2024-03-04T17:25:25.520 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-03-04T17:25:25.521 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_suriarte@teuthology --name suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi --archive /home/teuthworker/archive/suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi/7581361 --description rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} --verbose -- /home/teuthworker/archive/suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi/7581361/orig.config.yaml 2024-03-04T17:25:25.568 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T17:25:25.672 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi/7581361 branch: wip-rgw-d4n description: rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} email: null first_in_suite: false job_id: '7581361' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-rgw-d4n ceph: conf: client: debug rgw: 20 rgw bucket counters cache: true rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw d3n l1 datacache persistent path: /tmp/rgw_datacache/ rgw d3n l1 datacache size: 10737418240 rgw d3n l1 local datacache enabled: true rgw enable ops log: true rgw torrent flag: true rgw user counters cache: true setgroup: ceph setuser: ceph global: mon client directed command retry: 5 ms inject socket failures: 5000 osd heartbeat grace: 40 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 rgw override bucket index max shards: 1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd fast shutdown: false osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled - \(OSD_SLOW_PING_TIME sha1: 3aec390336fdfdc8f927818b718cfb0361bd47c8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3aec390336fdfdc8f927818b718cfb0361bd47c8 openssl_keys: rgw.client.0: ca: root client: client.0 embed-key: true root: client: client.0 cn: teuthology install: - client.0 key-type: rsa:4096 rgw: client.0: ssl certificate: rgw.client.0 valgrind: - --tool=memcheck - --max-threads=1024 compression type: random datacache: true datacache_path: /tmp/rgw_datacache ec-data-pool: false frontend: beast storage classes: LUKEWARM, FROZEN s3tests: force-branch: ceph-master storage classes: LUKEWARM, FROZEN thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rgw-d4n sha1: 3aec390336fdfdc8f927818b718cfb0361bd47c8 owner: scheduled_suriarte@teuthology priority: 150 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 3aec390336fdfdc8f927818b718cfb0361bd47c8 sleep_before_teardown: 0 suite: rgw suite_branch: wip-rgw-d4n suite_path: /home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa suite_relpath: qa suite_repo: https://github.com/pritha-srivastava/ceph.git suite_sha1: 3aec390336fdfdc8f927818b718cfb0361bd47c8 targets: smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwf7HDKkJ1igXST3tKMyYd6HtpysF747RRvrRt1UhlAM+YSj5Ofqp9m/IPxkJnFcDUuuMUvWFaaT2TvvPJGyk4= smithi173.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEDaCoKsQFWnbYcwH2ctXsZs8ZySlgfJZBOEhhmGWx4CEbqa+R0+ZUhWG8fplq/qksL24GS9lA3riaBlLKh5/aY= tasks: - install: extra_system_packages: deb: - s3cmd - maven rpm: - s3cmd - maven - ceph: null - openssl_keys: null - rgw: client.0: null - tox: - client.0 - workunit: clients: client.0: - rgw/run-datacache.sh env: RGW_DATACACHE_PATH: /tmp/rgw_datacache - workunit: clients: client.0: - rgw/run-bucket-check.sh - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - cls/test_cls_rgw_gc.sh - cls/test_cls_rgw_stats.sh - cls/test_cls_cmpomap.sh - cls/test_cls_2pc_queue.sh - rgw/test_rgw_gc_log.sh - rgw/test_rgw_obj.sh - rgw/test_librgw_file.sh - rgw/test_awssdkv4_sig.sh - workunit: clients: client.0: - rgw/test_rgw_s3_mp_reupload.sh - ragweed: client.0: default-branch: ceph-master rgw_server: client.0 stages: prepare,check - workunit: clients: client.0: - rgw/run-reshard.sh - s3tests-java: client.0: force-branch: ceph-master force-repo: https://github.com/ceph/java_s3tests.git - s3tests: client.0: rgw_server: client.0 - workunit: clients: client.0: - rgw/run-versioning.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-03-04_17:13:34 tube: smithi user: suriarte verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-04T17:25:25.672 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa; will attempt to use it 2024-03-04T17:25:25.673 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks 2024-03-04T17:25:25.673 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-04T17:25:25.675 INFO:teuthology.task.internal:Checking packages... 2024-03-04T17:25:25.700 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '3aec390336fdfdc8f927818b718cfb0361bd47c8' 2024-03-04T17:25:25.700 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-04T17:25:25.700 INFO:teuthology.packaging:ref: None 2024-03-04T17:25:25.700 INFO:teuthology.packaging:tag: None 2024-03-04T17:25:25.700 INFO:teuthology.packaging:branch: wip-rgw-d4n 2024-03-04T17:25:25.700 INFO:teuthology.packaging:sha1: 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:25:25.701 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-rgw-d4n 2024-03-04T17:25:25.972 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:25:25.973 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-04T17:25:25.981 INFO:teuthology.task.internal:no buildpackages task found 2024-03-04T17:25:25.981 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-04T17:25:26.000 INFO:teuthology.task.internal:Saving configuration 2024-03-04T17:25:26.018 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-04T17:25:26.025 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T17:25:26.052 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi/7581361', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 17:14:11.313070', 'locked_by': 'scheduled_suriarte@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEbkD2tTNMCsIrkXzYRynR6SncLw3HMLtaj66Uv2bnm5'} 2024-03-04T17:25:26.072 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi/7581361', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 17:14:11.314907', 'locked_by': 'scheduled_suriarte@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEZkH0otd5pc9QKDeJhFQSabQh57uSBuUeIogSSqgh9z'} 2024-03-04T17:25:26.073 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-04T17:25:26.079 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-03-04T17:25:26.079 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-03-04T17:25:26.079 INFO:teuthology.run_tasks:Running task console_log... 2024-03-04T17:25:26.159 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff2095afdc0>, signals=[15]) 2024-03-04T17:25:26.160 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-04T17:25:26.166 INFO:teuthology.task.internal:Opening connections... 2024-03-04T17:25:26.166 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-03-04T17:25:26.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:25:26.257 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2024-03-04T17:25:26.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:25:26.337 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-04T17:25:26.344 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-03-04T17:25:26.349 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-03-04T17:25:26.349 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-03-04T17:25:26.394 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-04T17:25:26.394 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="Ubuntu" 2024-03-04T17:25:26.394 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="22.04" 2024-03-04T17:25:26.394 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-04T17:25:26.394 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_CODENAME=jammy 2024-03-04T17:25:26.394 INFO:teuthology.orchestra.run.smithi137.stdout:ID=ubuntu 2024-03-04T17:25:26.394 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE=debian 2024-03-04T17:25:26.394 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-04T17:25:26.394 INFO:teuthology.orchestra.run.smithi137.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-04T17:25:26.394 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-04T17:25:26.394 INFO:teuthology.orchestra.run.smithi137.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-04T17:25:26.394 INFO:teuthology.orchestra.run.smithi137.stdout:UBUNTU_CODENAME=jammy 2024-03-04T17:25:26.395 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-03-04T17:25:26.423 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2024-03-04T17:25:26.429 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2024-03-04T17:25:26.429 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-03-04T17:25:26.477 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-04T17:25:26.477 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="Ubuntu" 2024-03-04T17:25:26.478 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="22.04" 2024-03-04T17:25:26.478 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-04T17:25:26.478 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_CODENAME=jammy 2024-03-04T17:25:26.478 INFO:teuthology.orchestra.run.smithi173.stdout:ID=ubuntu 2024-03-04T17:25:26.478 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE=debian 2024-03-04T17:25:26.478 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-04T17:25:26.478 INFO:teuthology.orchestra.run.smithi173.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-04T17:25:26.478 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-04T17:25:26.478 INFO:teuthology.orchestra.run.smithi173.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-04T17:25:26.478 INFO:teuthology.orchestra.run.smithi173.stdout:UBUNTU_CODENAME=jammy 2024-03-04T17:25:26.478 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-03-04T17:25:26.498 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-04T17:25:26.508 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-04T17:25:26.515 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-04T17:25:26.515 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-03-04T17:25:26.518 DEBUG:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2024-03-04T17:25:26.524 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-04T17:25:26.532 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-04T17:25:26.533 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-03-04T17:25:26.564 DEBUG:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2024-03-04T17:25:26.577 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-04T17:25:26.628 INFO:teuthology.run_tasks:Running task kernel... 2024-03-04T17:25:26.642 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-04T17:25:26.642 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-04T17:25:26.643 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T17:25:26.643 DEBUG:teuthology.orchestra.run.smithi173:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T17:25:26.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:25:26.648 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-03-04T17:25:26.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:25:26.649 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2024-03-04T17:25:26.695 INFO:teuthology.orchestra.run.smithi137.stdout:5.15.0-56-generic 2024-03-04T17:25:26.695 INFO:teuthology.task.kernel:Running kernel on smithi137: 5.15.0-56-generic 2024-03-04T17:25:26.696 DEBUG:teuthology.orchestra.run.smithi137:> sudo apt-get clean 2024-03-04T17:25:26.698 INFO:teuthology.orchestra.run.smithi173.stdout:5.15.0-56-generic 2024-03-04T17:25:26.698 INFO:teuthology.task.kernel:Running kernel on smithi173: 5.15.0-56-generic 2024-03-04T17:25:26.699 DEBUG:teuthology.orchestra.run.smithi173:> sudo apt-get clean 2024-03-04T17:25:26.818 DEBUG:teuthology.orchestra.run.smithi137:> sudo apt-get update 2024-03-04T17:25:26.899 DEBUG:teuthology.orchestra.run.smithi173:> sudo apt-get update 2024-03-04T17:25:27.020 INFO:teuthology.orchestra.run.smithi137.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-04T17:25:27.043 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T17:25:27.047 INFO:teuthology.orchestra.run.smithi173.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-04T17:25:27.052 INFO:teuthology.orchestra.run.smithi173.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-04T17:25:27.097 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T17:25:27.145 INFO:teuthology.orchestra.run.smithi173.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-04T17:25:27.166 INFO:teuthology.orchestra.run.smithi137.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-04T17:25:27.278 INFO:teuthology.orchestra.run.smithi173.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [413 kB] 2024-03-04T17:25:27.322 INFO:teuthology.orchestra.run.smithi137.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,205 kB] 2024-03-04T17:25:27.354 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,422 kB] 2024-03-04T17:25:27.366 INFO:teuthology.orchestra.run.smithi173.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,205 kB] 2024-03-04T17:25:27.445 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [219 kB] 2024-03-04T17:25:27.451 INFO:teuthology.orchestra.run.smithi173.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-04T17:25:27.452 INFO:teuthology.orchestra.run.smithi173.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-04T17:25:27.453 INFO:teuthology.orchestra.run.smithi173.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,476 kB] 2024-03-04T17:25:27.525 INFO:teuthology.orchestra.run.smithi173.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [244 kB] 2024-03-04T17:25:27.530 INFO:teuthology.orchestra.run.smithi137.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-04T17:25:27.531 INFO:teuthology.orchestra.run.smithi173.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-04T17:25:27.532 INFO:teuthology.orchestra.run.smithi173.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [596 kB] 2024-03-04T17:25:27.539 INFO:teuthology.orchestra.run.smithi137.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [413 kB] 2024-03-04T17:25:27.554 INFO:teuthology.orchestra.run.smithi137.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [219 kB] 2024-03-04T17:25:27.560 INFO:teuthology.orchestra.run.smithi137.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-04T17:25:27.561 INFO:teuthology.orchestra.run.smithi137.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,476 kB] 2024-03-04T17:25:27.570 INFO:teuthology.orchestra.run.smithi173.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [846 kB] 2024-03-04T17:25:27.586 INFO:teuthology.orchestra.run.smithi173.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [579 kB] 2024-03-04T17:25:27.651 INFO:teuthology.orchestra.run.smithi173.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [161 kB] 2024-03-04T17:25:27.654 INFO:teuthology.orchestra.run.smithi173.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-04T17:25:27.655 INFO:teuthology.orchestra.run.smithi173.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-04T17:25:27.655 INFO:teuthology.orchestra.run.smithi173.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [279 kB] 2024-03-04T17:25:27.655 INFO:teuthology.orchestra.run.smithi173.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-04T17:25:27.656 INFO:teuthology.orchestra.run.smithi173.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-04T17:25:27.657 INFO:teuthology.orchestra.run.smithi173.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-04T17:25:27.661 INFO:teuthology.orchestra.run.smithi173.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-04T17:25:27.661 INFO:teuthology.orchestra.run.smithi173.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,504 kB] 2024-03-04T17:25:27.664 INFO:teuthology.orchestra.run.smithi137.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-04T17:25:27.665 INFO:teuthology.orchestra.run.smithi137.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [244 kB] 2024-03-04T17:25:27.667 INFO:teuthology.orchestra.run.smithi137.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [579 kB] 2024-03-04T17:25:27.667 INFO:teuthology.orchestra.run.smithi137.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-04T17:25:27.668 INFO:teuthology.orchestra.run.smithi137.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [846 kB] 2024-03-04T17:25:27.735 INFO:teuthology.orchestra.run.smithi137.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [596 kB] 2024-03-04T17:25:27.739 INFO:teuthology.orchestra.run.smithi173.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-04T17:25:27.741 INFO:teuthology.orchestra.run.smithi173.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [247 kB] 2024-03-04T17:25:27.750 INFO:teuthology.orchestra.run.smithi137.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [161 kB] 2024-03-04T17:25:27.752 INFO:teuthology.orchestra.run.smithi173.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-04T17:25:27.752 INFO:teuthology.orchestra.run.smithi173.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,052 kB] 2024-03-04T17:25:27.759 INFO:teuthology.orchestra.run.smithi137.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-04T17:25:27.759 INFO:teuthology.orchestra.run.smithi137.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-04T17:25:27.760 INFO:teuthology.orchestra.run.smithi137.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-04T17:25:27.767 INFO:teuthology.orchestra.run.smithi137.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-04T17:25:27.767 INFO:teuthology.orchestra.run.smithi137.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-04T17:25:27.860 INFO:teuthology.orchestra.run.smithi137.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,422 kB] 2024-03-04T17:25:27.862 INFO:teuthology.orchestra.run.smithi173.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [691 kB] 2024-03-04T17:25:27.874 INFO:teuthology.orchestra.run.smithi173.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [237 kB] 2024-03-04T17:25:27.879 INFO:teuthology.orchestra.run.smithi173.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-04T17:25:27.880 INFO:teuthology.orchestra.run.smithi173.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-04T17:25:27.880 INFO:teuthology.orchestra.run.smithi173.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-04T17:25:27.882 INFO:teuthology.orchestra.run.smithi173.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-04T17:25:27.882 INFO:teuthology.orchestra.run.smithi173.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-04T17:25:27.882 INFO:teuthology.orchestra.run.smithi173.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2024-03-04T17:25:27.883 INFO:teuthology.orchestra.run.smithi173.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2024-03-04T17:25:27.899 INFO:teuthology.orchestra.run.smithi173.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2024-03-04T17:25:27.937 INFO:teuthology.orchestra.run.smithi173.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-04T17:25:27.938 INFO:teuthology.orchestra.run.smithi173.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2024-03-04T17:25:27.938 INFO:teuthology.orchestra.run.smithi173.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2024-03-04T17:25:27.939 INFO:teuthology.orchestra.run.smithi173.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-03-04T17:25:27.940 INFO:teuthology.orchestra.run.smithi173.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-04T17:25:28.003 INFO:teuthology.orchestra.run.smithi137.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [279 kB] 2024-03-04T17:25:28.019 INFO:teuthology.orchestra.run.smithi137.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-04T17:25:28.020 INFO:teuthology.orchestra.run.smithi137.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,504 kB] 2024-03-04T17:25:28.078 INFO:teuthology.orchestra.run.smithi137.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-04T17:25:28.079 INFO:teuthology.orchestra.run.smithi137.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [247 kB] 2024-03-04T17:25:28.085 INFO:teuthology.orchestra.run.smithi137.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-04T17:25:28.251 INFO:teuthology.orchestra.run.smithi137.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [691 kB] 2024-03-04T17:25:28.832 INFO:teuthology.orchestra.run.smithi137.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,052 kB] 2024-03-04T17:25:28.943 INFO:teuthology.orchestra.run.smithi137.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [237 kB] 2024-03-04T17:25:28.963 INFO:teuthology.orchestra.run.smithi137.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-04T17:25:28.964 INFO:teuthology.orchestra.run.smithi137.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-04T17:25:28.967 INFO:teuthology.orchestra.run.smithi137.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-04T17:25:28.967 INFO:teuthology.orchestra.run.smithi137.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-04T17:25:28.968 INFO:teuthology.orchestra.run.smithi137.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-04T17:25:28.969 INFO:teuthology.orchestra.run.smithi137.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2024-03-04T17:25:28.970 INFO:teuthology.orchestra.run.smithi137.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2024-03-04T17:25:28.974 INFO:teuthology.orchestra.run.smithi137.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2024-03-04T17:25:29.138 INFO:teuthology.orchestra.run.smithi137.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-04T17:25:29.221 INFO:teuthology.orchestra.run.smithi137.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2024-03-04T17:25:29.303 INFO:teuthology.orchestra.run.smithi137.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2024-03-04T17:25:29.355 INFO:teuthology.orchestra.run.smithi137.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-03-04T17:25:29.410 INFO:teuthology.orchestra.run.smithi137.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-04T17:25:32.866 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 11.9 MB in 2s (6,829 kB/s) 2024-03-04T17:25:33.759 INFO:teuthology.orchestra.run.smithi137.stdout:Fetched 11.9 MB in 3s (4,720 kB/s) 2024-03-04T17:25:34.191 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-03-04T17:25:34.209 DEBUG:teuthology.orchestra.run.smithi173:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-04T17:25:34.302 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-03-04T17:25:34.475 INFO:teuthology.orchestra.run.smithi173.stdout:Building dependency tree... 2024-03-04T17:25:34.475 INFO:teuthology.orchestra.run.smithi173.stdout:Reading state information... 2024-03-04T17:25:34.627 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:25:34.627 INFO:teuthology.orchestra.run.smithi173.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:25:34.627 INFO:teuthology.orchestra.run.smithi173.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:25:34.628 INFO:teuthology.orchestra.run.smithi173.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:25:34.628 INFO:teuthology.orchestra.run.smithi173.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-04T17:25:34.628 INFO:teuthology.orchestra.run.smithi173.stdout: python2.7-minimal 2024-03-04T17:25:34.628 INFO:teuthology.orchestra.run.smithi173.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:25:34.628 INFO:teuthology.orchestra.run.smithi173.stdout:The following additional packages will be installed: 2024-03-04T17:25:34.629 INFO:teuthology.orchestra.run.smithi173.stdout: linux-generic linux-headers-5.15.0-97 linux-headers-5.15.0-97-generic 2024-03-04T17:25:34.629 INFO:teuthology.orchestra.run.smithi173.stdout: linux-headers-generic linux-image-5.15.0-97-generic 2024-03-04T17:25:34.629 INFO:teuthology.orchestra.run.smithi173.stdout: linux-modules-5.15.0-97-generic linux-modules-extra-5.15.0-97-generic 2024-03-04T17:25:34.630 INFO:teuthology.orchestra.run.smithi173.stdout:Suggested packages: 2024-03-04T17:25:34.630 INFO:teuthology.orchestra.run.smithi173.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-04T17:25:34.630 INFO:teuthology.orchestra.run.smithi173.stdout:Recommended packages: 2024-03-04T17:25:34.630 INFO:teuthology.orchestra.run.smithi173.stdout: thermald 2024-03-04T17:25:34.672 INFO:teuthology.orchestra.run.smithi173.stdout:The following NEW packages will be installed: 2024-03-04T17:25:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: linux-headers-5.15.0-97 linux-headers-5.15.0-97-generic 2024-03-04T17:25:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: linux-image-5.15.0-97-generic linux-modules-5.15.0-97-generic 2024-03-04T17:25:34.672 INFO:teuthology.orchestra.run.smithi173.stdout: linux-modules-extra-5.15.0-97-generic 2024-03-04T17:25:34.673 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages will be upgraded: 2024-03-04T17:25:34.674 INFO:teuthology.orchestra.run.smithi173.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-04T17:25:34.728 INFO:teuthology.orchestra.run.smithi173.stdout:3 upgraded, 5 newly installed, 0 to remove and 288 not upgraded. 2024-03-04T17:25:34.728 INFO:teuthology.orchestra.run.smithi173.stdout:Need to get 113 MB of archives. 2024-03-04T17:25:34.729 INFO:teuthology.orchestra.run.smithi173.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-04T17:25:34.729 INFO:teuthology.orchestra.run.smithi173.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-97-generic amd64 5.15.0-97.107 [22.6 MB] 2024-03-04T17:25:35.100 INFO:teuthology.orchestra.run.smithi137.stdout:Reading package lists... 2024-03-04T17:25:35.118 DEBUG:teuthology.orchestra.run.smithi137:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-04T17:25:35.164 INFO:teuthology.orchestra.run.smithi173.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-97-generic amd64 5.15.0-97.107 [11.5 MB] 2024-03-04T17:25:35.181 INFO:teuthology.orchestra.run.smithi137.stdout:Reading package lists... 2024-03-04T17:25:35.292 INFO:teuthology.orchestra.run.smithi173.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-97-generic amd64 5.15.0-97.107 [63.8 MB] 2024-03-04T17:25:35.361 INFO:teuthology.orchestra.run.smithi137.stdout:Building dependency tree... 2024-03-04T17:25:35.361 INFO:teuthology.orchestra.run.smithi137.stdout:Reading state information... 2024-03-04T17:25:35.508 INFO:teuthology.orchestra.run.smithi137.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:25:35.509 INFO:teuthology.orchestra.run.smithi137.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:25:35.509 INFO:teuthology.orchestra.run.smithi137.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:25:35.509 INFO:teuthology.orchestra.run.smithi137.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:25:35.509 INFO:teuthology.orchestra.run.smithi137.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-04T17:25:35.509 INFO:teuthology.orchestra.run.smithi137.stdout: python2.7-minimal 2024-03-04T17:25:35.509 INFO:teuthology.orchestra.run.smithi137.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:25:35.510 INFO:teuthology.orchestra.run.smithi137.stdout:The following additional packages will be installed: 2024-03-04T17:25:35.510 INFO:teuthology.orchestra.run.smithi137.stdout: linux-generic linux-headers-5.15.0-97 linux-headers-5.15.0-97-generic 2024-03-04T17:25:35.510 INFO:teuthology.orchestra.run.smithi137.stdout: linux-headers-generic linux-image-5.15.0-97-generic 2024-03-04T17:25:35.511 INFO:teuthology.orchestra.run.smithi137.stdout: linux-modules-5.15.0-97-generic linux-modules-extra-5.15.0-97-generic 2024-03-04T17:25:35.512 INFO:teuthology.orchestra.run.smithi137.stdout:Suggested packages: 2024-03-04T17:25:35.512 INFO:teuthology.orchestra.run.smithi137.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-04T17:25:35.512 INFO:teuthology.orchestra.run.smithi137.stdout:Recommended packages: 2024-03-04T17:25:35.512 INFO:teuthology.orchestra.run.smithi137.stdout: thermald 2024-03-04T17:25:35.564 INFO:teuthology.orchestra.run.smithi137.stdout:The following NEW packages will be installed: 2024-03-04T17:25:35.564 INFO:teuthology.orchestra.run.smithi137.stdout: linux-headers-5.15.0-97 linux-headers-5.15.0-97-generic 2024-03-04T17:25:35.564 INFO:teuthology.orchestra.run.smithi137.stdout: linux-image-5.15.0-97-generic linux-modules-5.15.0-97-generic 2024-03-04T17:25:35.565 INFO:teuthology.orchestra.run.smithi137.stdout: linux-modules-extra-5.15.0-97-generic 2024-03-04T17:25:35.566 INFO:teuthology.orchestra.run.smithi137.stdout:The following packages will be upgraded: 2024-03-04T17:25:35.566 INFO:teuthology.orchestra.run.smithi137.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-04T17:25:35.662 INFO:teuthology.orchestra.run.smithi137.stdout:3 upgraded, 5 newly installed, 0 to remove and 288 not upgraded. 2024-03-04T17:25:35.662 INFO:teuthology.orchestra.run.smithi137.stdout:Need to get 113 MB of archives. 2024-03-04T17:25:35.662 INFO:teuthology.orchestra.run.smithi137.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-04T17:25:35.662 INFO:teuthology.orchestra.run.smithi137.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-97-generic amd64 5.15.0-97.107 [22.6 MB] 2024-03-04T17:25:36.496 INFO:teuthology.orchestra.run.smithi137.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-97-generic amd64 5.15.0-97.107 [11.5 MB] 2024-03-04T17:25:36.649 INFO:teuthology.orchestra.run.smithi137.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-97-generic amd64 5.15.0-97.107 [63.8 MB] 2024-03-04T17:25:36.765 INFO:teuthology.orchestra.run.smithi173.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.97.92 [1,696 B] 2024-03-04T17:25:36.765 INFO:teuthology.orchestra.run.smithi173.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.97.92 [2,494 B] 2024-03-04T17:25:36.765 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-97 all 5.15.0-97.107 [12.3 MB] 2024-03-04T17:25:37.173 INFO:teuthology.orchestra.run.smithi173.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-97-generic amd64 5.15.0-97.107 [2,892 kB] 2024-03-04T17:25:37.272 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.97.92 [2,342 B] 2024-03-04T17:25:37.666 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 113 MB in 3s (43.6 MB/s) 2024-03-04T17:25:37.680 INFO:teuthology.orchestra.run.smithi137.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.97.92 [1,696 B] 2024-03-04T17:25:37.680 INFO:teuthology.orchestra.run.smithi137.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.97.92 [2,494 B] 2024-03-04T17:25:37.680 INFO:teuthology.orchestra.run.smithi137.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-97 all 5.15.0-97.107 [12.3 MB] 2024-03-04T17:25:37.816 INFO:teuthology.orchestra.run.smithi137.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-97-generic amd64 5.15.0-97.107 [2,892 kB] 2024-03-04T17:25:37.841 INFO:teuthology.orchestra.run.smithi137.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.97.92 [2,342 B] 2024-03-04T17:25:37.853 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package linux-modules-5.15.0-97-generic. 2024-03-04T17:25:38.311 INFO:teuthology.orchestra.run.smithi137.stdout:Fetched 113 MB in 2s (48.4 MB/s) 2024-03-04T17:25:38.493 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package linux-modules-5.15.0-97-generic. 2024-03-04T17:25:39.839 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-04T17:25:39.843 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../0-linux-modules-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:25:39.916 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-modules-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:25:40.489 INFO:teuthology.orchestra.run.smithi137.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-04T17:25:40.493 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../0-linux-modules-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:25:40.565 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-modules-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:25:41.926 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package linux-image-5.15.0-97-generic. 2024-03-04T17:25:41.947 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../1-linux-image-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:25:42.022 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:25:42.578 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-97-generic. 2024-03-04T17:25:42.599 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:25:42.608 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package linux-image-5.15.0-97-generic. 2024-03-04T17:25:42.629 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../1-linux-image-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:25:42.637 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-modules-extra-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:25:42.723 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:25:43.619 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-97-generic. 2024-03-04T17:25:43.640 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:25:43.677 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-modules-extra-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:25:49.355 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../3-linux-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T17:25:49.454 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T17:25:49.762 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T17:25:49.891 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-image-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T17:25:50.019 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../3-linux-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T17:25:50.145 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T17:25:50.177 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package linux-headers-5.15.0-97. 2024-03-04T17:25:50.200 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../5-linux-headers-5.15.0-97_5.15.0-97.107_all.deb ... 2024-03-04T17:25:50.235 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-headers-5.15.0-97 (5.15.0-97.107) ... 2024-03-04T17:25:50.477 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T17:25:50.573 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-image-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T17:25:50.851 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package linux-headers-5.15.0-97. 2024-03-04T17:25:50.873 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../5-linux-headers-5.15.0-97_5.15.0-97.107_all.deb ... 2024-03-04T17:25:50.917 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-headers-5.15.0-97 (5.15.0-97.107) ... 2024-03-04T17:25:55.992 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package linux-headers-5.15.0-97-generic. 2024-03-04T17:25:56.028 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../6-linux-headers-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:25:56.067 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-headers-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:25:56.742 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package linux-headers-5.15.0-97-generic. 2024-03-04T17:25:56.780 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../6-linux-headers-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T17:25:56.817 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-headers-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:25:58.652 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T17:25:58.777 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-headers-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T17:25:59.094 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-headers-5.15.0-97 (5.15.0-97.107) ... 2024-03-04T17:25:59.207 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-headers-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:25:59.333 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-headers-generic (5.15.0.97.92) ... 2024-03-04T17:25:59.379 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T17:25:59.462 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:25:59.502 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-headers-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T17:25:59.837 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-headers-5.15.0-97 (5.15.0-97.107) ... 2024-03-04T17:25:59.966 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-headers-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:00.100 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-headers-generic (5.15.0.97.92) ... 2024-03-04T17:26:00.203 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:00.785 INFO:teuthology.orchestra.run.smithi173.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-04T17:26:00.786 INFO:teuthology.orchestra.run.smithi173.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-04T17:26:00.786 INFO:teuthology.orchestra.run.smithi173.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-97-generic 2024-03-04T17:26:00.786 INFO:teuthology.orchestra.run.smithi173.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-97-generic 2024-03-04T17:26:00.971 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-modules-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:01.513 INFO:teuthology.orchestra.run.smithi137.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-04T17:26:01.513 INFO:teuthology.orchestra.run.smithi137.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-04T17:26:01.514 INFO:teuthology.orchestra.run.smithi137.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-97-generic 2024-03-04T17:26:01.514 INFO:teuthology.orchestra.run.smithi137.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-97-generic 2024-03-04T17:26:01.672 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-modules-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:02.932 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-modules-extra-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:03.324 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-modules-extra-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:04.508 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-image-generic (5.15.0.97.92) ... 2024-03-04T17:26:04.626 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-generic (5.15.0.97.92) ... 2024-03-04T17:26:04.708 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-image-generic (5.15.0.97.92) ... 2024-03-04T17:26:04.769 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:04.809 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-generic (5.15.0.97.92) ... 2024-03-04T17:26:04.830 INFO:teuthology.orchestra.run.smithi173.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-04T17:26:04.830 INFO:teuthology.orchestra.run.smithi173.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-97-generic 2024-03-04T17:26:04.927 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T17:26:04.992 INFO:teuthology.orchestra.run.smithi137.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-04T17:26:04.992 INFO:teuthology.orchestra.run.smithi137.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-97-generic 2024-03-04T17:26:19.230 INFO:teuthology.orchestra.run.smithi173.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-04T17:26:19.230 INFO:teuthology.orchestra.run.smithi173.stdout:Sourcing file `/etc/default/grub' 2024-03-04T17:26:19.245 INFO:teuthology.orchestra.run.smithi173.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-04T17:26:19.269 INFO:teuthology.orchestra.run.smithi173.stdout:Generating grub configuration file ... 2024-03-04T17:26:19.541 INFO:teuthology.orchestra.run.smithi173.stdout:Found linux image: /boot/vmlinuz-5.15.0-97-generic 2024-03-04T17:26:19.564 INFO:teuthology.orchestra.run.smithi173.stdout:Found initrd image: /boot/initrd.img-5.15.0-97-generic 2024-03-04T17:26:19.766 INFO:teuthology.orchestra.run.smithi137.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-04T17:26:19.766 INFO:teuthology.orchestra.run.smithi137.stdout:Sourcing file `/etc/default/grub' 2024-03-04T17:26:19.774 INFO:teuthology.orchestra.run.smithi137.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-04T17:26:19.789 INFO:teuthology.orchestra.run.smithi137.stdout:Generating grub configuration file ... 2024-03-04T17:26:19.828 INFO:teuthology.orchestra.run.smithi173.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-04T17:26:19.833 INFO:teuthology.orchestra.run.smithi173.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-04T17:26:19.900 INFO:teuthology.orchestra.run.smithi173.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-04T17:26:19.903 INFO:teuthology.orchestra.run.smithi173.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-04T17:26:20.078 INFO:teuthology.orchestra.run.smithi173.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-04T17:26:20.078 INFO:teuthology.orchestra.run.smithi173.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-04T17:26:20.078 INFO:teuthology.orchestra.run.smithi173.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-04T17:26:20.127 INFO:teuthology.orchestra.run.smithi173.stdout:done 2024-03-04T17:26:20.144 INFO:teuthology.orchestra.run.smithi137.stdout:Found linux image: /boot/vmlinuz-5.15.0-97-generic 2024-03-04T17:26:20.172 INFO:teuthology.orchestra.run.smithi137.stdout:Found initrd image: /boot/initrd.img-5.15.0-97-generic 2024-03-04T17:26:20.553 INFO:teuthology.orchestra.run.smithi137.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-04T17:26:20.559 INFO:teuthology.orchestra.run.smithi137.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-04T17:26:20.601 DEBUG:teuthology.orchestra.run.smithi173:> dpkg -s linux-image-generic 2024-03-04T17:26:20.623 INFO:teuthology.orchestra.run.smithi173.stdout:Package: linux-image-generic 2024-03-04T17:26:20.624 INFO:teuthology.orchestra.run.smithi173.stdout:Status: install ok installed 2024-03-04T17:26:20.624 INFO:teuthology.orchestra.run.smithi173.stdout:Priority: optional 2024-03-04T17:26:20.624 INFO:teuthology.orchestra.run.smithi173.stdout:Section: kernel 2024-03-04T17:26:20.624 INFO:teuthology.orchestra.run.smithi173.stdout:Installed-Size: 21 2024-03-04T17:26:20.624 INFO:teuthology.orchestra.run.smithi173.stdout:Maintainer: Ubuntu Kernel Team 2024-03-04T17:26:20.624 INFO:teuthology.orchestra.run.smithi173.stdout:Architecture: amd64 2024-03-04T17:26:20.624 INFO:teuthology.orchestra.run.smithi173.stdout:Source: linux-meta 2024-03-04T17:26:20.624 INFO:teuthology.orchestra.run.smithi173.stdout:Version: 5.15.0.97.92 2024-03-04T17:26:20.624 INFO:teuthology.orchestra.run.smithi173.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-97), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-04T17:26:20.624 INFO:teuthology.orchestra.run.smithi173.stdout:Depends: linux-image-5.15.0-97-generic, linux-modules-extra-5.15.0-97-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-04T17:26:20.624 INFO:teuthology.orchestra.run.smithi173.stdout:Recommends: thermald 2024-03-04T17:26:20.624 INFO:teuthology.orchestra.run.smithi173.stdout:Description: Generic Linux kernel image 2024-03-04T17:26:20.625 INFO:teuthology.orchestra.run.smithi173.stdout: This package will always depend on the latest generic kernel image 2024-03-04T17:26:20.625 INFO:teuthology.orchestra.run.smithi173.stdout: available. 2024-03-04T17:26:20.625 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-97-generic 2024-03-04T17:26:20.625 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-04T17:26:20.625 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-04T17:26:20.625 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-04T17:26:20.625 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi173.front.sepia.ceph.com, path=None, version=distro) 2024-03-04T17:26:20.625 DEBUG:teuthology.orchestra.run.smithi173:> sudo apt-get clean 2024-03-04T17:26:20.626 INFO:teuthology.orchestra.run.smithi137.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-04T17:26:20.630 INFO:teuthology.orchestra.run.smithi137.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-04T17:26:20.784 DEBUG:teuthology.orchestra.run.smithi173:> sudo apt-get update 2024-03-04T17:26:20.789 INFO:teuthology.orchestra.run.smithi137.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-04T17:26:20.789 INFO:teuthology.orchestra.run.smithi137.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-04T17:26:20.789 INFO:teuthology.orchestra.run.smithi137.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-04T17:26:20.837 INFO:teuthology.orchestra.run.smithi137.stdout:done 2024-03-04T17:26:20.903 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T17:26:20.915 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-04T17:26:20.935 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-04T17:26:21.015 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-04T17:26:21.302 DEBUG:teuthology.orchestra.run.smithi137:> dpkg -s linux-image-generic 2024-03-04T17:26:21.341 INFO:teuthology.orchestra.run.smithi137.stdout:Package: linux-image-generic 2024-03-04T17:26:21.341 INFO:teuthology.orchestra.run.smithi137.stdout:Status: install ok installed 2024-03-04T17:26:21.341 INFO:teuthology.orchestra.run.smithi137.stdout:Priority: optional 2024-03-04T17:26:21.342 INFO:teuthology.orchestra.run.smithi137.stdout:Section: kernel 2024-03-04T17:26:21.342 INFO:teuthology.orchestra.run.smithi137.stdout:Installed-Size: 21 2024-03-04T17:26:21.342 INFO:teuthology.orchestra.run.smithi137.stdout:Maintainer: Ubuntu Kernel Team 2024-03-04T17:26:21.342 INFO:teuthology.orchestra.run.smithi137.stdout:Architecture: amd64 2024-03-04T17:26:21.342 INFO:teuthology.orchestra.run.smithi137.stdout:Source: linux-meta 2024-03-04T17:26:21.342 INFO:teuthology.orchestra.run.smithi137.stdout:Version: 5.15.0.97.92 2024-03-04T17:26:21.342 INFO:teuthology.orchestra.run.smithi137.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-97), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-04T17:26:21.342 INFO:teuthology.orchestra.run.smithi137.stdout:Depends: linux-image-5.15.0-97-generic, linux-modules-extra-5.15.0-97-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-04T17:26:21.342 INFO:teuthology.orchestra.run.smithi137.stdout:Recommends: thermald 2024-03-04T17:26:21.342 INFO:teuthology.orchestra.run.smithi137.stdout:Description: Generic Linux kernel image 2024-03-04T17:26:21.342 INFO:teuthology.orchestra.run.smithi137.stdout: This package will always depend on the latest generic kernel image 2024-03-04T17:26:21.342 INFO:teuthology.orchestra.run.smithi137.stdout: available. 2024-03-04T17:26:21.343 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-97-generic 2024-03-04T17:26:21.343 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-04T17:26:21.343 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-04T17:26:21.343 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-04T17:26:21.343 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi137.front.sepia.ceph.com, path=None, version=distro) 2024-03-04T17:26:21.343 DEBUG:teuthology.orchestra.run.smithi137:> sudo apt-get clean 2024-03-04T17:26:21.527 DEBUG:teuthology.orchestra.run.smithi137:> sudo apt-get update 2024-03-04T17:26:21.691 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T17:26:21.691 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-04T17:26:21.718 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-04T17:26:21.762 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-04T17:26:22.222 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-03-04T17:26:22.239 DEBUG:teuthology.orchestra.run.smithi173:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-04T17:26:22.300 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-03-04T17:26:22.496 INFO:teuthology.orchestra.run.smithi173.stdout:Building dependency tree... 2024-03-04T17:26:22.496 INFO:teuthology.orchestra.run.smithi173.stdout:Reading state information... 2024-03-04T17:26:22.670 INFO:teuthology.orchestra.run.smithi173.stdout:linux-image-generic is already the newest version (5.15.0.97.92). 2024-03-04T17:26:22.670 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:26:22.670 INFO:teuthology.orchestra.run.smithi173.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:26:22.670 INFO:teuthology.orchestra.run.smithi173.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:26:22.670 INFO:teuthology.orchestra.run.smithi173.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:26:22.670 INFO:teuthology.orchestra.run.smithi173.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T17:26:22.671 INFO:teuthology.orchestra.run.smithi173.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T17:26:22.671 INFO:teuthology.orchestra.run.smithi173.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:26:22.711 INFO:teuthology.orchestra.run.smithi173.stdout:0 upgraded, 0 newly installed, 0 to remove and 288 not upgraded. 2024-03-04T17:26:22.713 DEBUG:teuthology.orchestra.run.smithi173:> dpkg -s linux-image-generic 2024-03-04T17:26:22.729 INFO:teuthology.orchestra.run.smithi173.stdout:Package: linux-image-generic 2024-03-04T17:26:22.729 INFO:teuthology.orchestra.run.smithi173.stdout:Status: install ok installed 2024-03-04T17:26:22.729 INFO:teuthology.orchestra.run.smithi173.stdout:Priority: optional 2024-03-04T17:26:22.729 INFO:teuthology.orchestra.run.smithi173.stdout:Section: kernel 2024-03-04T17:26:22.729 INFO:teuthology.orchestra.run.smithi173.stdout:Installed-Size: 21 2024-03-04T17:26:22.729 INFO:teuthology.orchestra.run.smithi173.stdout:Maintainer: Ubuntu Kernel Team 2024-03-04T17:26:22.729 INFO:teuthology.orchestra.run.smithi173.stdout:Architecture: amd64 2024-03-04T17:26:22.729 INFO:teuthology.orchestra.run.smithi173.stdout:Source: linux-meta 2024-03-04T17:26:22.730 INFO:teuthology.orchestra.run.smithi173.stdout:Version: 5.15.0.97.92 2024-03-04T17:26:22.730 INFO:teuthology.orchestra.run.smithi173.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-97), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-04T17:26:22.730 INFO:teuthology.orchestra.run.smithi173.stdout:Depends: linux-image-5.15.0-97-generic, linux-modules-extra-5.15.0-97-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-04T17:26:22.730 INFO:teuthology.orchestra.run.smithi173.stdout:Recommends: thermald 2024-03-04T17:26:22.730 INFO:teuthology.orchestra.run.smithi173.stdout:Description: Generic Linux kernel image 2024-03-04T17:26:22.730 INFO:teuthology.orchestra.run.smithi173.stdout: This package will always depend on the latest generic kernel image 2024-03-04T17:26:22.730 INFO:teuthology.orchestra.run.smithi173.stdout: available. 2024-03-04T17:26:22.730 DEBUG:teuthology.orchestra.run.smithi173:> mktemp 2024-03-04T17:26:22.776 INFO:teuthology.orchestra.run.smithi173.stdout:/tmp/tmp.TGGNmGFmyr 2024-03-04T17:26:22.776 DEBUG:teuthology.orchestra.run.smithi173:> sudo cp /boot/grub/grub.cfg /tmp/tmp.TGGNmGFmyr 2024-03-04T17:26:22.828 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0666 /tmp/tmp.TGGNmGFmyr 2024-03-04T17:26:22.951 DEBUG:teuthology.orchestra.remote:smithi173:/tmp/tmp.TGGNmGFmyr is 10KB 2024-03-04T17:26:22.965 DEBUG:teuthology.orchestra.run.smithi173:> rm -fr /tmp/tmp.TGGNmGFmyr 2024-03-04T17:26:22.976 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-04T17:26:23.029 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:26:23.029 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-04T17:26:23.029 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-04T17:26:23.060 INFO:teuthology.orchestra.run.smithi137.stdout:Reading package lists... 2024-03-04T17:26:23.077 DEBUG:teuthology.orchestra.run.smithi137:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-04T17:26:23.097 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-97-generic 2024-03-04T17:26:23.097 DEBUG:teuthology.orchestra.run.smithi173:> sudo update-grub 2024-03-04T17:26:23.129 INFO:teuthology.orchestra.run.smithi137.stdout:Reading package lists... 2024-03-04T17:26:23.318 INFO:teuthology.orchestra.run.smithi137.stdout:Building dependency tree... 2024-03-04T17:26:23.319 INFO:teuthology.orchestra.run.smithi137.stdout:Reading state information... 2024-03-04T17:26:23.470 INFO:teuthology.orchestra.run.smithi137.stdout:linux-image-generic is already the newest version (5.15.0.97.92). 2024-03-04T17:26:23.471 INFO:teuthology.orchestra.run.smithi137.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:26:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:26:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:26:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:26:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T17:26:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T17:26:23.472 INFO:teuthology.orchestra.run.smithi137.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:26:23.509 INFO:teuthology.orchestra.run.smithi137.stdout:0 upgraded, 0 newly installed, 0 to remove and 288 not upgraded. 2024-03-04T17:26:23.511 DEBUG:teuthology.orchestra.run.smithi137:> dpkg -s linux-image-generic 2024-03-04T17:26:23.569 INFO:teuthology.orchestra.run.smithi137.stdout:Package: linux-image-generic 2024-03-04T17:26:23.569 INFO:teuthology.orchestra.run.smithi137.stdout:Status: install ok installed 2024-03-04T17:26:23.569 INFO:teuthology.orchestra.run.smithi137.stdout:Priority: optional 2024-03-04T17:26:23.569 INFO:teuthology.orchestra.run.smithi137.stdout:Section: kernel 2024-03-04T17:26:23.569 INFO:teuthology.orchestra.run.smithi137.stdout:Installed-Size: 21 2024-03-04T17:26:23.569 INFO:teuthology.orchestra.run.smithi137.stdout:Maintainer: Ubuntu Kernel Team 2024-03-04T17:26:23.569 INFO:teuthology.orchestra.run.smithi137.stdout:Architecture: amd64 2024-03-04T17:26:23.569 INFO:teuthology.orchestra.run.smithi137.stdout:Source: linux-meta 2024-03-04T17:26:23.569 INFO:teuthology.orchestra.run.smithi137.stdout:Version: 5.15.0.97.92 2024-03-04T17:26:23.569 INFO:teuthology.orchestra.run.smithi137.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-97), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-04T17:26:23.569 INFO:teuthology.orchestra.run.smithi137.stdout:Depends: linux-image-5.15.0-97-generic, linux-modules-extra-5.15.0-97-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-04T17:26:23.569 INFO:teuthology.orchestra.run.smithi137.stdout:Recommends: thermald 2024-03-04T17:26:23.570 INFO:teuthology.orchestra.run.smithi137.stdout:Description: Generic Linux kernel image 2024-03-04T17:26:23.570 INFO:teuthology.orchestra.run.smithi137.stdout: This package will always depend on the latest generic kernel image 2024-03-04T17:26:23.570 INFO:teuthology.orchestra.run.smithi137.stdout: available. 2024-03-04T17:26:23.570 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-03-04T17:26:23.614 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.KZkcCfRGqX 2024-03-04T17:26:23.614 DEBUG:teuthology.orchestra.run.smithi137:> sudo cp /boot/grub/grub.cfg /tmp/tmp.KZkcCfRGqX 2024-03-04T17:26:23.670 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.KZkcCfRGqX 2024-03-04T17:26:23.795 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.KZkcCfRGqX is 10KB 2024-03-04T17:26:23.813 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.KZkcCfRGqX 2024-03-04T17:26:23.818 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-04T17:26:23.874 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:26:23.874 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-04T17:26:23.874 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-04T17:26:23.945 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-97-generic 2024-03-04T17:26:23.945 DEBUG:teuthology.orchestra.run.smithi137:> sudo update-grub 2024-03-04T17:26:24.264 INFO:teuthology.orchestra.run.smithi173.stderr:Sourcing file `/etc/default/grub' 2024-03-04T17:26:24.265 INFO:teuthology.orchestra.run.smithi173.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-04T17:26:24.267 INFO:teuthology.orchestra.run.smithi173.stderr:Generating grub configuration file ... 2024-03-04T17:26:24.447 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-5.15.0-97-generic 2024-03-04T17:26:24.452 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-5.15.0-97-generic 2024-03-04T17:26:24.685 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-04T17:26:24.690 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-04T17:26:24.729 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-04T17:26:24.734 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-04T17:26:24.846 INFO:teuthology.orchestra.run.smithi173.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-04T17:26:24.846 INFO:teuthology.orchestra.run.smithi173.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-04T17:26:24.846 INFO:teuthology.orchestra.run.smithi173.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-04T17:26:24.867 INFO:teuthology.orchestra.run.smithi173.stderr:done 2024-03-04T17:26:24.879 DEBUG:teuthology.orchestra.run.smithi173:> sudo shutdown -r now 2024-03-04T17:26:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:Sourcing file `/etc/default/grub' 2024-03-04T17:26:25.122 INFO:teuthology.orchestra.run.smithi137.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-04T17:26:25.124 INFO:teuthology.orchestra.run.smithi137.stderr:Generating grub configuration file ... 2024-03-04T17:26:25.324 INFO:teuthology.orchestra.run.smithi137.stderr:Found linux image: /boot/vmlinuz-5.15.0-97-generic 2024-03-04T17:26:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:Found initrd image: /boot/initrd.img-5.15.0-97-generic 2024-03-04T17:26:25.599 INFO:teuthology.orchestra.run.smithi137.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-04T17:26:25.604 INFO:teuthology.orchestra.run.smithi137.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-04T17:26:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-04T17:26:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-04T17:26:25.776 INFO:teuthology.orchestra.run.smithi137.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-04T17:26:25.776 INFO:teuthology.orchestra.run.smithi137.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-04T17:26:25.776 INFO:teuthology.orchestra.run.smithi137.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-04T17:26:25.790 INFO:teuthology.orchestra.run.smithi137.stderr:done 2024-03-04T17:26:25.799 DEBUG:teuthology.orchestra.run.smithi137:> sudo shutdown -r now 2024-03-04T17:26:54.908 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-04T17:26:54.909 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-03-04T17:26:54.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:26:55.803 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-04T17:26:55.804 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-04T17:26:55.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:27:54.968 DEBUG:teuthology.orchestra.remote:timed out 2024-03-04T17:27:55.808 DEBUG:teuthology.orchestra.remote:timed out 2024-03-04T17:28:03.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-03-04T17:28:03.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:28:03.978 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-03-04T17:28:04.810 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-04T17:28:04.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:28:05.345 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-03-04T17:28:06.345 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-04T17:28:06.345 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-97-generic"... 2024-03-04T17:28:06.346 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-03-04T17:28:06.392 INFO:teuthology.orchestra.run.smithi137.stdout:5.15.0-97-generic 2024-03-04T17:28:06.392 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-97-generic vs 5.15.0-97-generic 2024-03-04T17:28:06.392 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-04T17:28:06.392 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-04T17:28:07.394 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-04T17:28:07.394 DEBUG:teuthology.orchestra.run.smithi137:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-04T17:28:07.823 INFO:teuthology.orchestra.run.smithi137.stdout:ttyS1 2024-03-04T17:28:07.839 DEBUG:teuthology.parallel:result is None 2024-03-04T17:28:15.979 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-03-04T17:28:15.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:28:16.228 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-03-04T17:28:16.752 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-03-04T17:28:16.752 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-97-generic"... 2024-03-04T17:28:16.752 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2024-03-04T17:28:16.798 INFO:teuthology.orchestra.run.smithi173.stdout:5.15.0-97-generic 2024-03-04T17:28:16.798 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-97-generic vs 5.15.0-97-generic 2024-03-04T17:28:16.798 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-04T17:28:16.799 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-04T17:28:17.800 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-04T17:28:17.800 DEBUG:teuthology.orchestra.run.smithi173:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-04T17:28:17.884 INFO:teuthology.orchestra.run.smithi173.stdout:ttyS1 2024-03-04T17:28:17.912 DEBUG:teuthology.parallel:result is None 2024-03-04T17:28:17.913 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-04T17:28:17.920 INFO:teuthology.task.internal:Creating test directory... 2024-03-04T17:28:17.923 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T17:28:17.926 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T17:28:17.931 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-04T17:28:17.939 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-04T17:28:17.950 INFO:teuthology.task.internal:Creating archive directory... 2024-03-04T17:28:17.951 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T17:28:17.973 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T17:28:18.070 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-04T17:28:18.078 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-04T17:28:18.078 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T17:28:18.086 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T17:28:18.102 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T17:28:18.104 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T17:28:18.110 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T17:28:18.112 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T17:28:18.114 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-04T17:28:18.121 INFO:teuthology.task.internal:Configuring sudo... 2024-03-04T17:28:18.122 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T17:28:18.157 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T17:28:18.181 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-04T17:28:18.192 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-04T17:28:18.192 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T17:28:18.225 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T17:28:18.230 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T17:28:18.276 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T17:28:18.324 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:28:18.324 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T17:28:18.394 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T17:28:18.400 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T17:28:18.447 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:28:18.448 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T17:28:18.505 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-03-04T17:28:18.508 DEBUG:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2024-03-04T17:28:18.578 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-04T17:28:18.586 INFO:teuthology.task.internal:Starting timer... 2024-03-04T17:28:18.586 INFO:teuthology.run_tasks:Running task pcp... 2024-03-04T17:28:18.597 INFO:teuthology.run_tasks:Running task selinux... 2024-03-04T17:28:18.608 DEBUG:teuthology.task.selinux:Excluding smithi137: OS 'ubuntu' does not support SELinux 2024-03-04T17:28:18.609 DEBUG:teuthology.task.selinux:Excluding smithi173: OS 'ubuntu' does not support SELinux 2024-03-04T17:28:18.609 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-04T17:28:18.609 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-04T17:28:18.609 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-04T17:28:18.609 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-04T17:28:18.622 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-04T17:28:18.646 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-04T17:28:18.648 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi137.front.sepia.ceph.com,smithi173.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-04T17:36:09.103 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi137.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com')] 2024-03-04T17:36:09.105 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-04T17:36:09.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:36:09.186 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-03-04T17:36:09.260 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-04T17:36:09.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-03-04T17:36:09.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T17:36:09.342 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-03-04T17:36:09.420 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-03-04T17:36:09.423 INFO:teuthology.run_tasks:Running task clock... 2024-03-04T17:36:09.434 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-04T17:36:09.434 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T17:36:09.434 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T17:36:09.437 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T17:36:09.437 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T17:36:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-04T17:36:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: Command line: ntpd -gq 2024-03-04T17:36:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: ---------------------------------------------------- 2024-03-04T17:36:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: ntp-4 is maintained by Network Time Foundation, 2024-03-04T17:36:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-04T17:36:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: corporation. Support and training for ntp-4 are 2024-03-04T17:36:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: available at https://www.nwtime.org/support 2024-03-04T17:36:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: ---------------------------------------------------- 2024-03-04T17:36:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: proto: precision = 0.043 usec (-24) 2024-03-04T17:36:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: basedate set to 2022-02-04 2024-03-04T17:36:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: gps base set to 2022-02-06 (week 2196) 2024-03-04T17:36:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T17:36:09.465 INFO:teuthology.orchestra.run.smithi137.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T17:36:09.465 INFO:teuthology.orchestra.run.smithi137.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-04T17:36:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: restrict ::: KOD does nothing without LIMITED. 2024-03-04T17:36:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: Listen and drop on 0 v6wildcard [::]:123 2024-03-04T17:36:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-04T17:36:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: Listen normally on 2 lo 127.0.0.1:123 2024-03-04T17:36:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: Listen normally on 3 enp3s0f1 172.21.15.137:123 2024-03-04T17:36:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: Listen normally on 4 lo [::1]:123 2024-03-04T17:36:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8137%5]:123 2024-03-04T17:36:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:09 ntpd[17178]: Listening on routing socket on fd #22 for interface updates 2024-03-04T17:36:09.490 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-04T17:36:09.490 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: Command line: ntpd -gq 2024-03-04T17:36:09.490 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: ---------------------------------------------------- 2024-03-04T17:36:09.490 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: ntp-4 is maintained by Network Time Foundation, 2024-03-04T17:36:09.490 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-04T17:36:09.490 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: corporation. Support and training for ntp-4 are 2024-03-04T17:36:09.490 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: available at https://www.nwtime.org/support 2024-03-04T17:36:09.490 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: ---------------------------------------------------- 2024-03-04T17:36:09.491 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: proto: precision = 0.043 usec (-24) 2024-03-04T17:36:09.491 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: basedate set to 2022-02-04 2024-03-04T17:36:09.491 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: gps base set to 2022-02-06 (week 2196) 2024-03-04T17:36:09.491 INFO:teuthology.orchestra.run.smithi173.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T17:36:09.492 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T17:36:09.492 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: restrict ::: KOD does nothing without LIMITED. 2024-03-04T17:36:09.492 INFO:teuthology.orchestra.run.smithi173.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-04T17:36:09.493 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: Listen and drop on 0 v6wildcard [::]:123 2024-03-04T17:36:09.493 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-04T17:36:09.493 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: Listen normally on 2 lo 127.0.0.1:123 2024-03-04T17:36:09.493 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: Listen normally on 3 enp3s0f1 172.21.15.173:123 2024-03-04T17:36:09.493 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: Listen normally on 4 lo [::1]:123 2024-03-04T17:36:09.493 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7259%5]:123 2024-03-04T17:36:09.493 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:09 ntpd[17164]: Listening on routing socket on fd #22 for interface updates 2024-03-04T17:36:10.464 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:10 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:10.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:10 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:10.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:10 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:10.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:10 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:10.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:10 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:10.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:10 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:10.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:10 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:10.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:10 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:10.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:10 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:10.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:10 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:10.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:10 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:10.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:10 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:11.464 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:11 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:11.464 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:11 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:11.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:11 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:11.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:11 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:11.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:11 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:11.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:11 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:11.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:11 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:11.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:11 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:12.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:12 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:12.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:12 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:12.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:12 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:12.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:12 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:12.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:12 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:12.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:12 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:12.466 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:12 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:12.466 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:12 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:12.466 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:12 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:12.466 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:12 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:12.466 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:12 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:12.466 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:12 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:12.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:12 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:12.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:12 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:12.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:12 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:12.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:12 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:12.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:12 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:12.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:12 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:12.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:12 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:12.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:12 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:13.464 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:13 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:13.464 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:13 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:13.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:13 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:13.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:13 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:13.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:13 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:13.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:13 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:13.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:13 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:13.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:13 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:13.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:13 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:13.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:13 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:13.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:13 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:13.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:13 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:14.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:14 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:14.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:14 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:14.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:14 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:14.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:14 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:14.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:14 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:14.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:14 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:14.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:14 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:14.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:14 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:14.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:14 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:14.466 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:14 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:14.466 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:14 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:14.466 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:14 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:14.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:14 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:14.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:14 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:14.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:14 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:14.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:14 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:14.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:14 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:14.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:14 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:14.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:14 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:14.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:14 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:14.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:14 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:14.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:14 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:14.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:14 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:14.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:14 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:15.464 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:15 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:15.464 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:15 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:15.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:15 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:15.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:15 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:15.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:15 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:15.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:15 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:15.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:15 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:15.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:15 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:15.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:15 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:15.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:15 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:15.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:15 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:15.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:15 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:16.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:16 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:16.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:16 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:16.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:16 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:16.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:16 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:16.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:16 ntpd[17178]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:16.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:16 ntpd[17178]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:16.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:16 ntpd[17178]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:16.465 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:16 ntpd[17178]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:16.466 INFO:teuthology.orchestra.run.smithi137.stdout: 4 Mar 17:36:16 ntpd[17178]: ntpd: time slew -0.000694 s 2024-03-04T17:36:16.466 INFO:teuthology.orchestra.run.smithi137.stdout:ntpd: time slew -0.000694s 2024-03-04T17:36:16.466 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:16 ntpd[17178]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-04T17:36:16.466 INFO:teuthology.orchestra.run.smithi137.stderr: 4 Mar 17:36:16 ntpd[17178]: can't open /var/log/ntpstats/loopstats.20240304: Permission denied 2024-03-04T17:36:16.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:16 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:16.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:16 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:16.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:16 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:16.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:16 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:16.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:16 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:16.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:16 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:16.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:16 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:16.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:16 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:16.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:16 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:16.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:16 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:16.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:16 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:16.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:16 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:16.515 INFO:teuthology.orchestra.run.smithi137.stdout: remote refid st t when poll reach delay offset jitter 2024-03-04T17:36:16.515 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================== 2024-03-04T17:36:16.515 INFO:teuthology.orchestra.run.smithi137.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:36:16.515 INFO:teuthology.orchestra.run.smithi137.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:36:16.515 INFO:teuthology.orchestra.run.smithi137.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:36:16.515 INFO:teuthology.orchestra.run.smithi137.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:36:17.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:17 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:17.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:17 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:17.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:17 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T17:36:17.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:17 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T17:36:17.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:17 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T17:36:17.491 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:17 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T17:36:17.492 INFO:teuthology.orchestra.run.smithi173.stdout: 4 Mar 17:36:17 ntpd[17164]: ntpd: time slew -0.001037 s 2024-03-04T17:36:17.492 INFO:teuthology.orchestra.run.smithi173.stdout:ntpd: time slew -0.001037s 2024-03-04T17:36:17.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:17 ntpd[17164]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-04T17:36:17.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4 Mar 17:36:17 ntpd[17164]: can't open /var/log/ntpstats/loopstats.20240304: Permission denied 2024-03-04T17:36:17.602 INFO:teuthology.orchestra.run.smithi173.stdout: remote refid st t when poll reach delay offset jitter 2024-03-04T17:36:17.603 INFO:teuthology.orchestra.run.smithi173.stdout:============================================================================== 2024-03-04T17:36:17.603 INFO:teuthology.orchestra.run.smithi173.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:36:17.603 INFO:teuthology.orchestra.run.smithi173.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:36:17.603 INFO:teuthology.orchestra.run.smithi173.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:36:17.603 INFO:teuthology.orchestra.run.smithi173.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T17:36:17.604 INFO:teuthology.run_tasks:Running task install... 2024-03-04T17:36:17.613 DEBUG:teuthology.task.install:project ceph 2024-03-04T17:36:17.613 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8'}} 2024-03-04T17:36:17.613 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'flavor': 'default', 'sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8'} 2024-03-04T17:36:17.613 INFO:teuthology.task.install:Using flavor: default 2024-03-04T17:36:17.619 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-04T17:36:17.619 INFO:teuthology.task.install:extra packages: [] 2024-03-04T17:36:17.619 DEBUG:teuthology.orchestra.run.smithi137:> sudo apt-key list | grep Ceph 2024-03-04T17:36:17.620 DEBUG:teuthology.orchestra.run.smithi173:> sudo apt-key list | grep Ceph 2024-03-04T17:36:17.684 INFO:teuthology.orchestra.run.smithi137.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-04T17:36:17.703 INFO:teuthology.orchestra.run.smithi173.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-04T17:36:17.713 INFO:teuthology.orchestra.run.smithi137.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-04T17:36:17.713 INFO:teuthology.orchestra.run.smithi137.stdout:uid [ unknown] Ceph.com (release key) 2024-03-04T17:36:17.714 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-03-04T17:36:17.714 INFO:teuthology.task.install.deb:Installing system (non-project) packages: s3cmd, maven on remote deb x86_64 2024-03-04T17:36:17.715 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:36:17.730 INFO:teuthology.orchestra.run.smithi173.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-04T17:36:17.730 INFO:teuthology.orchestra.run.smithi173.stdout:uid [ unknown] Ceph.com (release key) 2024-03-04T17:36:17.732 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-03-04T17:36:17.732 INFO:teuthology.task.install.deb:Installing system (non-project) packages: s3cmd, maven on remote deb x86_64 2024-03-04T17:36:17.732 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:36:17.860 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default/ 2024-03-04T17:36:17.860 INFO:teuthology.task.install.deb:Package version is 19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:36:18.012 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default/ 2024-03-04T17:36:18.012 INFO:teuthology.task.install.deb:Package version is 19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:36:18.060 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:36:18.061 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-04T17:36:18.115 DEBUG:teuthology.orchestra.run.smithi137:> sudo apt-get update 2024-03-04T17:36:18.228 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:36:18.228 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-04T17:36:18.242 DEBUG:teuthology.orchestra.run.smithi173:> sudo apt-get update 2024-03-04T17:36:18.290 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T17:36:18.304 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-04T17:36:18.325 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-04T17:36:18.380 INFO:teuthology.orchestra.run.smithi137.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy InRelease 2024-03-04T17:36:18.412 INFO:teuthology.orchestra.run.smithi137.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-04T17:36:18.415 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T17:36:18.415 INFO:teuthology.orchestra.run.smithi173.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-04T17:36:18.427 INFO:teuthology.orchestra.run.smithi173.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-04T17:36:18.428 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-04T17:36:18.445 INFO:teuthology.orchestra.run.smithi137.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-04T17:36:18.478 INFO:teuthology.orchestra.run.smithi137.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,395 B] 2024-03-04T17:36:18.509 INFO:teuthology.orchestra.run.smithi173.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy InRelease 2024-03-04T17:36:18.512 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-04T17:36:18.516 INFO:teuthology.orchestra.run.smithi137.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-04T17:36:18.542 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-04T17:36:18.575 INFO:teuthology.orchestra.run.smithi173.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-04T17:36:18.607 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,395 B] 2024-03-04T17:36:18.643 INFO:teuthology.orchestra.run.smithi137.stdout:Fetched 28.9 kB in 0s (82.3 kB/s) 2024-03-04T17:36:18.647 INFO:teuthology.orchestra.run.smithi173.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-04T17:36:18.768 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 258 kB in 0s (742 kB/s) 2024-03-04T17:36:19.591 INFO:teuthology.orchestra.run.smithi137.stdout:Reading package lists... 2024-03-04T17:36:19.607 DEBUG:teuthology.orchestra.run.smithi137:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1871-g3aec3903-1jammy cephadm=19.0.0-1871-g3aec3903-1jammy ceph-mds=19.0.0-1871-g3aec3903-1jammy ceph-mgr=19.0.0-1871-g3aec3903-1jammy ceph-common=19.0.0-1871-g3aec3903-1jammy ceph-fuse=19.0.0-1871-g3aec3903-1jammy ceph-test=19.0.0-1871-g3aec3903-1jammy ceph-volume=19.0.0-1871-g3aec3903-1jammy radosgw=19.0.0-1871-g3aec3903-1jammy python3-rados=19.0.0-1871-g3aec3903-1jammy python3-rgw=19.0.0-1871-g3aec3903-1jammy python3-cephfs=19.0.0-1871-g3aec3903-1jammy python3-rbd=19.0.0-1871-g3aec3903-1jammy libcephfs2=19.0.0-1871-g3aec3903-1jammy libcephfs-dev=19.0.0-1871-g3aec3903-1jammy librados2=19.0.0-1871-g3aec3903-1jammy librbd1=19.0.0-1871-g3aec3903-1jammy rbd-fuse=19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:36:19.667 INFO:teuthology.orchestra.run.smithi137.stdout:Reading package lists... 2024-03-04T17:36:19.674 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-03-04T17:36:19.691 DEBUG:teuthology.orchestra.run.smithi173:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1871-g3aec3903-1jammy cephadm=19.0.0-1871-g3aec3903-1jammy ceph-mds=19.0.0-1871-g3aec3903-1jammy ceph-mgr=19.0.0-1871-g3aec3903-1jammy ceph-common=19.0.0-1871-g3aec3903-1jammy ceph-fuse=19.0.0-1871-g3aec3903-1jammy ceph-test=19.0.0-1871-g3aec3903-1jammy ceph-volume=19.0.0-1871-g3aec3903-1jammy radosgw=19.0.0-1871-g3aec3903-1jammy python3-rados=19.0.0-1871-g3aec3903-1jammy python3-rgw=19.0.0-1871-g3aec3903-1jammy python3-cephfs=19.0.0-1871-g3aec3903-1jammy python3-rbd=19.0.0-1871-g3aec3903-1jammy libcephfs2=19.0.0-1871-g3aec3903-1jammy libcephfs-dev=19.0.0-1871-g3aec3903-1jammy librados2=19.0.0-1871-g3aec3903-1jammy librbd1=19.0.0-1871-g3aec3903-1jammy rbd-fuse=19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:36:19.753 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-03-04T17:36:19.859 INFO:teuthology.orchestra.run.smithi137.stdout:Building dependency tree... 2024-03-04T17:36:19.860 INFO:teuthology.orchestra.run.smithi137.stdout:Reading state information... 2024-03-04T17:36:19.946 INFO:teuthology.orchestra.run.smithi173.stdout:Building dependency tree... 2024-03-04T17:36:19.947 INFO:teuthology.orchestra.run.smithi173.stdout:Reading state information... 2024-03-04T17:36:20.041 INFO:teuthology.orchestra.run.smithi137.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:36:20.042 INFO:teuthology.orchestra.run.smithi137.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:36:20.042 INFO:teuthology.orchestra.run.smithi137.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:36:20.042 INFO:teuthology.orchestra.run.smithi137.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:36:20.042 INFO:teuthology.orchestra.run.smithi137.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T17:36:20.042 INFO:teuthology.orchestra.run.smithi137.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T17:36:20.042 INFO:teuthology.orchestra.run.smithi137.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:36:20.043 INFO:teuthology.orchestra.run.smithi137.stdout:The following additional packages will be installed: 2024-03-04T17:36:20.043 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-04T17:36:20.043 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-04T17:36:20.044 INFO:teuthology.orchestra.run.smithi137.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-04T17:36:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-04T17:36:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-04T17:36:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-04T17:36:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-04T17:36:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-04T17:36:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-04T17:36:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-04T17:36:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-04T17:36:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-04T17:36:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-04T17:36:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-04T17:36:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-04T17:36:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-04T17:36:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-03-04T17:36:20.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-03-04T17:36:20.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-04T17:36:20.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-04T17:36:20.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-04T17:36:20.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-04T17:36:20.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-03-04T17:36:20.046 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet zip 2024-03-04T17:36:20.046 INFO:teuthology.orchestra.run.smithi137.stdout:Suggested packages: 2024-03-04T17:36:20.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-04T17:36:20.046 INFO:teuthology.orchestra.run.smithi137.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-04T17:36:20.047 INFO:teuthology.orchestra.run.smithi137.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-04T17:36:20.047 INFO:teuthology.orchestra.run.smithi137.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-04T17:36:20.047 INFO:teuthology.orchestra.run.smithi137.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-04T17:36:20.047 INFO:teuthology.orchestra.run.smithi137.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-04T17:36:20.047 INFO:teuthology.orchestra.run.smithi137.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-04T17:36:20.047 INFO:teuthology.orchestra.run.smithi137.stdout:Recommended packages: 2024-03-04T17:36:20.047 INFO:teuthology.orchestra.run.smithi137.stdout: btrfs-tools 2024-03-04T17:36:20.127 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:36:20.128 INFO:teuthology.orchestra.run.smithi173.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:36:20.128 INFO:teuthology.orchestra.run.smithi173.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:36:20.128 INFO:teuthology.orchestra.run.smithi173.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:36:20.128 INFO:teuthology.orchestra.run.smithi173.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T17:36:20.129 INFO:teuthology.orchestra.run.smithi173.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T17:36:20.129 INFO:teuthology.orchestra.run.smithi173.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:36:20.129 INFO:teuthology.orchestra.run.smithi173.stdout:The following additional packages will be installed: 2024-03-04T17:36:20.129 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-04T17:36:20.129 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-04T17:36:20.129 INFO:teuthology.orchestra.run.smithi173.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-04T17:36:20.130 INFO:teuthology.orchestra.run.smithi173.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-04T17:36:20.131 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-04T17:36:20.131 INFO:teuthology.orchestra.run.smithi173.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-04T17:36:20.131 INFO:teuthology.orchestra.run.smithi173.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-04T17:36:20.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-04T17:36:20.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-04T17:36:20.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-04T17:36:20.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-04T17:36:20.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-04T17:36:20.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-04T17:36:20.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-04T17:36:20.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-04T17:36:20.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-04T17:36:20.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-03-04T17:36:20.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-03-04T17:36:20.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-04T17:36:20.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-04T17:36:20.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-04T17:36:20.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-04T17:36:20.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-03-04T17:36:20.132 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet zip 2024-03-04T17:36:20.132 INFO:teuthology.orchestra.run.smithi173.stdout:Suggested packages: 2024-03-04T17:36:20.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-04T17:36:20.133 INFO:teuthology.orchestra.run.smithi173.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-04T17:36:20.133 INFO:teuthology.orchestra.run.smithi173.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-04T17:36:20.133 INFO:teuthology.orchestra.run.smithi173.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-04T17:36:20.133 INFO:teuthology.orchestra.run.smithi173.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-04T17:36:20.133 INFO:teuthology.orchestra.run.smithi173.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-04T17:36:20.133 INFO:teuthology.orchestra.run.smithi173.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-04T17:36:20.133 INFO:teuthology.orchestra.run.smithi173.stdout:Recommended packages: 2024-03-04T17:36:20.133 INFO:teuthology.orchestra.run.smithi173.stdout: btrfs-tools 2024-03-04T17:36:20.147 INFO:teuthology.orchestra.run.smithi137.stdout:The following NEW packages will be installed: 2024-03-04T17:36:20.147 INFO:teuthology.orchestra.run.smithi137.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-04T17:36:20.147 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-04T17:36:20.148 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-03-04T17:36:20.149 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-03-04T17:36:20.149 INFO:teuthology.orchestra.run.smithi137.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-03-04T17:36:20.149 INFO:teuthology.orchestra.run.smithi137.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-03-04T17:36:20.149 INFO:teuthology.orchestra.run.smithi137.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-03-04T17:36:20.149 INFO:teuthology.orchestra.run.smithi137.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-03-04T17:36:20.149 INFO:teuthology.orchestra.run.smithi137.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-03-04T17:36:20.149 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-03-04T17:36:20.149 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-03-04T17:36:20.149 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-04T17:36:20.149 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-04T17:36:20.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-04T17:36:20.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-04T17:36:20.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-04T17:36:20.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-04T17:36:20.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-04T17:36:20.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-03-04T17:36:20.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-03-04T17:36:20.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-03-04T17:36:20.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-03-04T17:36:20.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-03-04T17:36:20.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-03-04T17:36:20.150 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-03-04T17:36:20.150 INFO:teuthology.orchestra.run.smithi137.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-03-04T17:36:20.234 INFO:teuthology.orchestra.run.smithi173.stdout:The following NEW packages will be installed: 2024-03-04T17:36:20.234 INFO:teuthology.orchestra.run.smithi173.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-04T17:36:20.234 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-04T17:36:20.234 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-03-04T17:36:20.234 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-03-04T17:36:20.234 INFO:teuthology.orchestra.run.smithi173.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-03-04T17:36:20.234 INFO:teuthology.orchestra.run.smithi173.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-03-04T17:36:20.235 INFO:teuthology.orchestra.run.smithi173.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-03-04T17:36:20.236 INFO:teuthology.orchestra.run.smithi173.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-03-04T17:36:20.236 INFO:teuthology.orchestra.run.smithi173.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-03-04T17:36:20.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-03-04T17:36:20.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-03-04T17:36:20.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-04T17:36:20.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-04T17:36:20.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-04T17:36:20.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-04T17:36:20.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-04T17:36:20.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-04T17:36:20.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-04T17:36:20.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-03-04T17:36:20.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-03-04T17:36:20.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-03-04T17:36:20.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-03-04T17:36:20.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-03-04T17:36:20.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-03-04T17:36:20.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-03-04T17:36:20.237 INFO:teuthology.orchestra.run.smithi173.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-03-04T17:36:20.295 INFO:teuthology.orchestra.run.smithi137.stdout:0 upgraded, 118 newly installed, 0 to remove and 275 not upgraded. 2024-03-04T17:36:20.296 INFO:teuthology.orchestra.run.smithi137.stdout:Need to get 177 MB of archives. 2024-03-04T17:36:20.296 INFO:teuthology.orchestra.run.smithi137.stdout:After this operation, 734 MB of additional disk space will be used. 2024-03-04T17:36:20.296 INFO:teuthology.orchestra.run.smithi137.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1871-g3aec3903-1jammy [3,547 kB] 2024-03-04T17:36:20.299 INFO:teuthology.orchestra.run.smithi173.stdout:0 upgraded, 118 newly installed, 0 to remove and 275 not upgraded. 2024-03-04T17:36:20.299 INFO:teuthology.orchestra.run.smithi173.stdout:Need to get 177 MB of archives. 2024-03-04T17:36:20.299 INFO:teuthology.orchestra.run.smithi173.stdout:After this operation, 734 MB of additional disk space will be used. 2024-03-04T17:36:20.299 INFO:teuthology.orchestra.run.smithi173.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-04T17:36:20.335 INFO:teuthology.orchestra.run.smithi137.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-04T17:36:20.406 INFO:teuthology.orchestra.run.smithi173.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-04T17:36:20.408 INFO:teuthology.orchestra.run.smithi173.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-04T17:36:20.428 INFO:teuthology.orchestra.run.smithi173.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-04T17:36:20.453 INFO:teuthology.orchestra.run.smithi173.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1871-g3aec3903-1jammy [3,547 kB] 2024-03-04T17:36:20.505 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-04T17:36:20.509 INFO:teuthology.orchestra.run.smithi173.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-04T17:36:20.523 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-04T17:36:20.528 INFO:teuthology.orchestra.run.smithi173.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-04T17:36:20.529 INFO:teuthology.orchestra.run.smithi173.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-04T17:36:20.530 INFO:teuthology.orchestra.run.smithi173.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-04T17:36:20.541 INFO:teuthology.orchestra.run.smithi173.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-04T17:36:20.543 INFO:teuthology.orchestra.run.smithi173.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-04T17:36:20.546 INFO:teuthology.orchestra.run.smithi173.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-04T17:36:20.554 INFO:teuthology.orchestra.run.smithi173.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-04T17:36:20.581 INFO:teuthology.orchestra.run.smithi173.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-04T17:36:20.584 INFO:teuthology.orchestra.run.smithi173.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-04T17:36:20.588 INFO:teuthology.orchestra.run.smithi173.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-04T17:36:20.590 INFO:teuthology.orchestra.run.smithi173.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-04T17:36:20.591 INFO:teuthology.orchestra.run.smithi173.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-04T17:36:20.591 INFO:teuthology.orchestra.run.smithi173.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-04T17:36:20.593 INFO:teuthology.orchestra.run.smithi173.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-04T17:36:20.593 INFO:teuthology.orchestra.run.smithi173.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-04T17:36:20.594 INFO:teuthology.orchestra.run.smithi173.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-04T17:36:20.619 INFO:teuthology.orchestra.run.smithi137.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1871-g3aec3903-1jammy [3,217 kB] 2024-03-04T17:36:20.625 INFO:teuthology.orchestra.run.smithi173.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-04T17:36:20.626 INFO:teuthology.orchestra.run.smithi173.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-04T17:36:20.651 INFO:teuthology.orchestra.run.smithi173.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-04T17:36:20.652 INFO:teuthology.orchestra.run.smithi173.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-04T17:36:20.658 INFO:teuthology.orchestra.run.smithi173.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-04T17:36:20.659 INFO:teuthology.orchestra.run.smithi173.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-04T17:36:20.660 INFO:teuthology.orchestra.run.smithi173.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-04T17:36:20.661 INFO:teuthology.orchestra.run.smithi173.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-04T17:36:20.662 INFO:teuthology.orchestra.run.smithi173.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-04T17:36:20.663 INFO:teuthology.orchestra.run.smithi173.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-04T17:36:20.672 INFO:teuthology.orchestra.run.smithi173.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-04T17:36:20.677 INFO:teuthology.orchestra.run.smithi173.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-04T17:36:20.692 INFO:teuthology.orchestra.run.smithi173.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-04T17:36:20.693 INFO:teuthology.orchestra.run.smithi173.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-04T17:36:20.705 INFO:teuthology.orchestra.run.smithi173.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-04T17:36:20.706 INFO:teuthology.orchestra.run.smithi173.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-04T17:36:20.707 INFO:teuthology.orchestra.run.smithi173.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-04T17:36:20.708 INFO:teuthology.orchestra.run.smithi173.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-04T17:36:20.710 INFO:teuthology.orchestra.run.smithi173.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-04T17:36:20.714 INFO:teuthology.orchestra.run.smithi173.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-04T17:36:20.716 INFO:teuthology.orchestra.run.smithi173.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-04T17:36:20.719 INFO:teuthology.orchestra.run.smithi173.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-04T17:36:20.735 INFO:teuthology.orchestra.run.smithi173.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-04T17:36:20.741 INFO:teuthology.orchestra.run.smithi173.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-04T17:36:20.742 INFO:teuthology.orchestra.run.smithi173.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-04T17:36:20.744 INFO:teuthology.orchestra.run.smithi173.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-04T17:36:20.749 INFO:teuthology.orchestra.run.smithi173.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-04T17:36:20.762 INFO:teuthology.orchestra.run.smithi137.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-04T17:36:20.774 INFO:teuthology.orchestra.run.smithi137.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-04T17:36:20.808 INFO:teuthology.orchestra.run.smithi137.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1871-g3aec3903-1jammy [710 kB] 2024-03-04T17:36:20.838 INFO:teuthology.orchestra.run.smithi173.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-04T17:36:20.839 INFO:teuthology.orchestra.run.smithi173.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-04T17:36:20.841 INFO:teuthology.orchestra.run.smithi173.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-04T17:36:20.842 INFO:teuthology.orchestra.run.smithi173.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-04T17:36:20.844 INFO:teuthology.orchestra.run.smithi173.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-04T17:36:20.846 INFO:teuthology.orchestra.run.smithi137.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1871-g3aec3903-1jammy [356 kB] 2024-03-04T17:36:20.855 INFO:teuthology.orchestra.run.smithi137.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-04T17:36:20.863 INFO:teuthology.orchestra.run.smithi137.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1871-g3aec3903-1jammy [32.8 kB] 2024-03-04T17:36:20.864 INFO:teuthology.orchestra.run.smithi137.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1871-g3aec3903-1jammy [180 kB] 2024-03-04T17:36:20.873 INFO:teuthology.orchestra.run.smithi137.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1871-g3aec3903-1jammy [64.4 kB] 2024-03-04T17:36:20.875 INFO:teuthology.orchestra.run.smithi173.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-04T17:36:20.876 INFO:teuthology.orchestra.run.smithi137.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1871-g3aec3903-1jammy [330 kB] 2024-03-04T17:36:20.878 INFO:teuthology.orchestra.run.smithi173.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-04T17:36:20.880 INFO:teuthology.orchestra.run.smithi173.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-04T17:36:20.891 INFO:teuthology.orchestra.run.smithi137.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1871-g3aec3903-1jammy [6,634 kB] 2024-03-04T17:36:20.918 INFO:teuthology.orchestra.run.smithi173.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-04T17:36:20.918 INFO:teuthology.orchestra.run.smithi173.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-04T17:36:20.932 INFO:teuthology.orchestra.run.smithi173.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-04T17:36:20.933 INFO:teuthology.orchestra.run.smithi173.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-04T17:36:20.934 INFO:teuthology.orchestra.run.smithi173.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-04T17:36:20.935 INFO:teuthology.orchestra.run.smithi173.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-04T17:36:20.935 INFO:teuthology.orchestra.run.smithi173.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-04T17:36:20.936 INFO:teuthology.orchestra.run.smithi173.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-04T17:36:20.941 INFO:teuthology.orchestra.run.smithi173.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-04T17:36:20.953 INFO:teuthology.orchestra.run.smithi173.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-04T17:36:20.955 INFO:teuthology.orchestra.run.smithi173.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-04T17:36:20.956 INFO:teuthology.orchestra.run.smithi173.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-04T17:36:20.976 INFO:teuthology.orchestra.run.smithi173.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-04T17:36:20.979 INFO:teuthology.orchestra.run.smithi173.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-04T17:36:20.981 INFO:teuthology.orchestra.run.smithi173.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-04T17:36:20.981 INFO:teuthology.orchestra.run.smithi173.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-04T17:36:20.988 INFO:teuthology.orchestra.run.smithi173.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-04T17:36:20.989 INFO:teuthology.orchestra.run.smithi173.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-04T17:36:20.994 INFO:teuthology.orchestra.run.smithi173.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-04T17:36:20.998 INFO:teuthology.orchestra.run.smithi173.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-04T17:36:20.999 INFO:teuthology.orchestra.run.smithi173.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-04T17:36:21.000 INFO:teuthology.orchestra.run.smithi173.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-04T17:36:21.019 INFO:teuthology.orchestra.run.smithi173.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-04T17:36:21.020 INFO:teuthology.orchestra.run.smithi173.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-04T17:36:21.030 INFO:teuthology.orchestra.run.smithi173.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-04T17:36:21.030 INFO:teuthology.orchestra.run.smithi173.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-04T17:36:21.031 INFO:teuthology.orchestra.run.smithi173.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-04T17:36:21.034 INFO:teuthology.orchestra.run.smithi173.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-04T17:36:21.035 INFO:teuthology.orchestra.run.smithi173.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-04T17:36:21.085 INFO:teuthology.orchestra.run.smithi137.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-04T17:36:21.097 INFO:teuthology.orchestra.run.smithi137.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-04T17:36:21.126 INFO:teuthology.orchestra.run.smithi137.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-04T17:36:21.136 INFO:teuthology.orchestra.run.smithi137.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-04T17:36:21.136 INFO:teuthology.orchestra.run.smithi137.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-04T17:36:21.138 INFO:teuthology.orchestra.run.smithi137.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-04T17:36:21.157 INFO:teuthology.orchestra.run.smithi137.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-04T17:36:21.264 INFO:teuthology.orchestra.run.smithi137.stdout:Get:21 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1871-g3aec3903-1jammy [112 kB] 2024-03-04T17:36:21.269 INFO:teuthology.orchestra.run.smithi137.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1871-g3aec3903-1jammy [466 kB] 2024-03-04T17:36:21.298 INFO:teuthology.orchestra.run.smithi137.stdout:Get:23 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1871-g3aec3903-1jammy [24.2 MB] 2024-03-04T17:36:21.325 INFO:teuthology.orchestra.run.smithi137.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-04T17:36:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-04T17:36:21.772 INFO:teuthology.orchestra.run.smithi137.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-04T17:36:21.774 INFO:teuthology.orchestra.run.smithi137.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-04T17:36:21.854 INFO:teuthology.orchestra.run.smithi137.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-04T17:36:21.905 INFO:teuthology.orchestra.run.smithi137.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-04T17:36:21.927 INFO:teuthology.orchestra.run.smithi137.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-04T17:36:21.934 INFO:teuthology.orchestra.run.smithi137.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-04T17:36:21.935 INFO:teuthology.orchestra.run.smithi137.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-04T17:36:21.947 INFO:teuthology.orchestra.run.smithi137.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-04T17:36:21.947 INFO:teuthology.orchestra.run.smithi137.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-04T17:36:22.113 INFO:teuthology.orchestra.run.smithi137.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-04T17:36:22.196 INFO:teuthology.orchestra.run.smithi137.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-04T17:36:22.224 INFO:teuthology.orchestra.run.smithi137.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-04T17:36:22.241 INFO:teuthology.orchestra.run.smithi137.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-04T17:36:22.285 INFO:teuthology.orchestra.run.smithi137.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-04T17:36:22.432 INFO:teuthology.orchestra.run.smithi173.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1871-g3aec3903-1jammy [3,217 kB] 2024-03-04T17:36:22.496 INFO:teuthology.orchestra.run.smithi137.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-04T17:36:22.510 INFO:teuthology.orchestra.run.smithi137.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-04T17:36:22.518 INFO:teuthology.orchestra.run.smithi137.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-04T17:36:22.523 INFO:teuthology.orchestra.run.smithi137.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-04T17:36:22.544 INFO:teuthology.orchestra.run.smithi137.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-04T17:36:22.548 INFO:teuthology.orchestra.run.smithi137.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-04T17:36:22.673 INFO:teuthology.orchestra.run.smithi137.stdout:Get:46 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1871-g3aec3903-1jammy [5,077 kB] 2024-03-04T17:36:22.714 INFO:teuthology.orchestra.run.smithi137.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-04T17:36:22.884 INFO:teuthology.orchestra.run.smithi137.stdout:Get:48 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1871-g3aec3903-1jammy [243 kB] 2024-03-04T17:36:22.897 INFO:teuthology.orchestra.run.smithi137.stdout:Get:49 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1871-g3aec3903-1jammy [124 kB] 2024-03-04T17:36:22.905 INFO:teuthology.orchestra.run.smithi137.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1871-g3aec3903-1jammy [1,521 kB] 2024-03-04T17:36:22.996 INFO:teuthology.orchestra.run.smithi137.stdout:Get:51 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1871-g3aec3903-1jammy [6,237 kB] 2024-03-04T17:36:23.046 INFO:teuthology.orchestra.run.smithi137.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-04T17:36:23.078 INFO:teuthology.orchestra.run.smithi137.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-04T17:36:23.088 INFO:teuthology.orchestra.run.smithi137.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-04T17:36:23.272 INFO:teuthology.orchestra.run.smithi173.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1871-g3aec3903-1jammy [710 kB] 2024-03-04T17:36:23.290 INFO:teuthology.orchestra.run.smithi137.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-04T17:36:23.290 INFO:teuthology.orchestra.run.smithi137.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-04T17:36:23.297 INFO:teuthology.orchestra.run.smithi137.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-04T17:36:23.303 INFO:teuthology.orchestra.run.smithi137.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-04T17:36:23.318 INFO:teuthology.orchestra.run.smithi137.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-04T17:36:23.339 INFO:teuthology.orchestra.run.smithi137.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-04T17:36:23.345 INFO:teuthology.orchestra.run.smithi137.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-04T17:36:23.353 INFO:teuthology.orchestra.run.smithi137.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1871-g3aec3903-1jammy [22.7 MB] 2024-03-04T17:36:23.444 INFO:teuthology.orchestra.run.smithi173.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1871-g3aec3903-1jammy [356 kB] 2024-03-04T17:36:23.482 INFO:teuthology.orchestra.run.smithi173.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1871-g3aec3903-1jammy [32.8 kB] 2024-03-04T17:36:23.518 INFO:teuthology.orchestra.run.smithi173.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1871-g3aec3903-1jammy [180 kB] 2024-03-04T17:36:23.520 INFO:teuthology.orchestra.run.smithi137.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-04T17:36:23.549 INFO:teuthology.orchestra.run.smithi173.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1871-g3aec3903-1jammy [64.4 kB] 2024-03-04T17:36:23.556 INFO:teuthology.orchestra.run.smithi173.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1871-g3aec3903-1jammy [330 kB] 2024-03-04T17:36:23.640 INFO:teuthology.orchestra.run.smithi173.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1871-g3aec3903-1jammy [6,634 kB] 2024-03-04T17:36:23.784 INFO:teuthology.orchestra.run.smithi137.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-04T17:36:23.962 INFO:teuthology.orchestra.run.smithi137.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-04T17:36:23.976 INFO:teuthology.orchestra.run.smithi137.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-04T17:36:23.996 INFO:teuthology.orchestra.run.smithi137.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-04T17:36:24.061 INFO:teuthology.orchestra.run.smithi137.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-04T17:36:24.349 INFO:teuthology.orchestra.run.smithi137.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-04T17:36:24.352 INFO:teuthology.orchestra.run.smithi137.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-04T17:36:24.355 INFO:teuthology.orchestra.run.smithi137.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-04T17:36:24.356 INFO:teuthology.orchestra.run.smithi137.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-04T17:36:24.358 INFO:teuthology.orchestra.run.smithi137.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-04T17:36:24.579 INFO:teuthology.orchestra.run.smithi137.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-04T17:36:24.906 INFO:teuthology.orchestra.run.smithi137.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1871-g3aec3903-1jammy [14.0 kB] 2024-03-04T17:36:24.906 INFO:teuthology.orchestra.run.smithi137.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1871-g3aec3903-1jammy [866 kB] 2024-03-04T17:36:24.923 INFO:teuthology.orchestra.run.smithi137.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1871-g3aec3903-1jammy [2,190 kB] 2024-03-04T17:36:25.009 INFO:teuthology.orchestra.run.smithi173.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1871-g3aec3903-1jammy [112 kB] 2024-03-04T17:36:25.013 INFO:teuthology.orchestra.run.smithi137.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-04T17:36:25.018 INFO:teuthology.orchestra.run.smithi173.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1871-g3aec3903-1jammy [466 kB] 2024-03-04T17:36:25.019 INFO:teuthology.orchestra.run.smithi137.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-04T17:36:25.020 INFO:teuthology.orchestra.run.smithi137.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1871-g3aec3903-1jammy [777 kB] 2024-03-04T17:36:25.068 INFO:teuthology.orchestra.run.smithi137.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1871-g3aec3903-1jammy [144 kB] 2024-03-04T17:36:25.076 INFO:teuthology.orchestra.run.smithi137.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1871-g3aec3903-1jammy [1,922 kB] 2024-03-04T17:36:25.105 INFO:teuthology.orchestra.run.smithi173.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1871-g3aec3903-1jammy [24.2 MB] 2024-03-04T17:36:25.207 INFO:teuthology.orchestra.run.smithi137.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1871-g3aec3903-1jammy [8,625 kB] 2024-03-04T17:36:25.308 INFO:teuthology.orchestra.run.smithi137.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-04T17:36:25.309 INFO:teuthology.orchestra.run.smithi137.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-04T17:36:25.311 INFO:teuthology.orchestra.run.smithi137.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-04T17:36:25.313 INFO:teuthology.orchestra.run.smithi137.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-04T17:36:25.314 INFO:teuthology.orchestra.run.smithi137.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-04T17:36:25.317 INFO:teuthology.orchestra.run.smithi137.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-04T17:36:25.317 INFO:teuthology.orchestra.run.smithi137.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-04T17:36:25.319 INFO:teuthology.orchestra.run.smithi137.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-04T17:36:25.501 INFO:teuthology.orchestra.run.smithi137.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-04T17:36:25.794 INFO:teuthology.orchestra.run.smithi137.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1871-g3aec3903-1jammy [14.2 kB] 2024-03-04T17:36:25.795 INFO:teuthology.orchestra.run.smithi137.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1871-g3aec3903-1jammy [50.7 MB] 2024-03-04T17:36:25.906 INFO:teuthology.orchestra.run.smithi137.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-04T17:36:25.953 INFO:teuthology.orchestra.run.smithi137.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-04T17:36:25.972 INFO:teuthology.orchestra.run.smithi137.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-04T17:36:26.084 INFO:teuthology.orchestra.run.smithi137.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-04T17:36:26.124 INFO:teuthology.orchestra.run.smithi137.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-04T17:36:26.136 INFO:teuthology.orchestra.run.smithi137.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-04T17:36:26.140 INFO:teuthology.orchestra.run.smithi137.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-04T17:36:26.188 INFO:teuthology.orchestra.run.smithi137.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-04T17:36:26.191 INFO:teuthology.orchestra.run.smithi137.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-04T17:36:26.212 INFO:teuthology.orchestra.run.smithi137.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-04T17:36:26.376 INFO:teuthology.orchestra.run.smithi137.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-04T17:36:26.650 INFO:teuthology.orchestra.run.smithi137.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-04T17:36:26.667 INFO:teuthology.orchestra.run.smithi137.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-04T17:36:26.820 INFO:teuthology.orchestra.run.smithi137.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-04T17:36:26.850 INFO:teuthology.orchestra.run.smithi137.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-04T17:36:27.008 INFO:teuthology.orchestra.run.smithi137.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-04T17:36:27.011 INFO:teuthology.orchestra.run.smithi137.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-04T17:36:27.012 INFO:teuthology.orchestra.run.smithi137.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-04T17:36:27.035 INFO:teuthology.orchestra.run.smithi137.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-04T17:36:27.040 INFO:teuthology.orchestra.run.smithi137.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-04T17:36:30.922 INFO:teuthology.orchestra.run.smithi137.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1871-g3aec3903-1jammy [121 kB] 2024-03-04T17:36:30.924 INFO:teuthology.orchestra.run.smithi137.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1871-g3aec3903-1jammy [38.4 kB] 2024-03-04T17:36:30.925 INFO:teuthology.orchestra.run.smithi137.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1871-g3aec3903-1jammy [13.1 MB] 2024-03-04T17:36:31.331 INFO:teuthology.orchestra.run.smithi173.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1871-g3aec3903-1jammy [5,077 kB] 2024-03-04T17:36:33.227 INFO:teuthology.orchestra.run.smithi173.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1871-g3aec3903-1jammy [243 kB] 2024-03-04T17:36:33.331 INFO:teuthology.orchestra.run.smithi173.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1871-g3aec3903-1jammy [124 kB] 2024-03-04T17:36:33.397 INFO:teuthology.orchestra.run.smithi137.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1871-g3aec3903-1jammy [91.7 kB] 2024-03-04T17:36:33.403 INFO:teuthology.orchestra.run.smithi173.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1871-g3aec3903-1jammy [1,521 kB] 2024-03-04T17:36:33.957 INFO:teuthology.orchestra.run.smithi137.stdout:Fetched 177 MB in 13s (13.4 MB/s) 2024-03-04T17:36:34.084 INFO:teuthology.orchestra.run.smithi173.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1871-g3aec3903-1jammy [6,237 kB] 2024-03-04T17:36:34.088 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-04T17:36:34.167 INFO:teuthology.orchestra.run.smithi137.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165994 files and directories currently installed.) 2024-03-04T17:36:34.173 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-04T17:36:34.237 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T17:36:34.516 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-04T17:36:34.531 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-04T17:36:34.574 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T17:36:34.910 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-04T17:36:34.936 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-04T17:36:34.968 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T17:36:35.422 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-04T17:36:35.448 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T17:36:35.498 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:36:35.892 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-04T17:36:35.917 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T17:36:35.958 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:36:36.603 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-04T17:36:36.630 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T17:36:36.773 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:36:36.773 INFO:teuthology.orchestra.run.smithi173.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1871-g3aec3903-1jammy [22.7 MB] 2024-03-04T17:36:37.173 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-04T17:36:37.199 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-04T17:36:37.239 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T17:36:37.584 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package librados2. 2024-03-04T17:36:37.610 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../007-librados2_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:36:37.650 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking librados2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:36:38.078 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package librbd1. 2024-03-04T17:36:38.104 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../008-librbd1_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:36:38.153 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking librbd1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:36:38.580 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libcephfs2. 2024-03-04T17:36:38.607 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:36:38.647 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libcephfs2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:36:38.924 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-rados. 2024-03-04T17:36:38.951 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../010-python3-rados_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:36:38.991 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-rados (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:36:39.310 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-04T17:36:39.336 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:36:39.377 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-ceph-argparse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:36:39.629 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-cephfs. 2024-03-04T17:36:39.644 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:36:39.679 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-cephfs (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:36:39.957 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-04T17:36:39.983 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:36:40.024 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:36:40.352 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-04T17:36:40.378 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-04T17:36:40.418 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T17:36:40.738 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-prettytable. 2024-03-04T17:36:40.764 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-03-04T17:36:40.804 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-04T17:36:41.065 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-rbd. 2024-03-04T17:36:41.091 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:36:41.140 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-rbd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:36:41.443 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-04T17:36:41.469 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-04T17:36:41.509 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T17:36:41.954 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-04T17:36:41.977 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-04T17:36:42.012 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T17:36:42.290 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-04T17:36:42.316 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-04T17:36:42.357 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T17:36:42.693 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package lua5.1. 2024-03-04T17:36:42.719 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-04T17:36:42.760 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-04T17:36:43.088 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package lua-any. 2024-03-04T17:36:43.114 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-03-04T17:36:43.154 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-04T17:36:43.357 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package zip. 2024-03-04T17:36:43.372 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-03-04T17:36:43.407 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking zip (3.0-12build2) ... 2024-03-04T17:36:43.710 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package unzip. 2024-03-04T17:36:43.736 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-04T17:36:43.776 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-04T17:36:44.180 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package luarocks. 2024-03-04T17:36:44.206 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-04T17:36:44.247 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-04T17:36:44.575 INFO:teuthology.orchestra.run.smithi173.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1871-g3aec3903-1jammy [14.0 kB] 2024-03-04T17:36:44.576 INFO:teuthology.orchestra.run.smithi173.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1871-g3aec3903-1jammy [866 kB] 2024-03-04T17:36:44.630 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package librgw2. 2024-03-04T17:36:44.656 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../025-librgw2_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:36:44.686 INFO:teuthology.orchestra.run.smithi173.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1871-g3aec3903-1jammy [2,190 kB] 2024-03-04T17:36:44.713 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking librgw2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:36:45.130 INFO:teuthology.orchestra.run.smithi173.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1871-g3aec3903-1jammy [777 kB] 2024-03-04T17:36:45.256 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-rgw. 2024-03-04T17:36:45.283 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:36:45.323 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-rgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:36:45.369 INFO:teuthology.orchestra.run.smithi173.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1871-g3aec3903-1jammy [144 kB] 2024-03-04T17:36:45.372 INFO:teuthology.orchestra.run.smithi173.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1871-g3aec3903-1jammy [1,922 kB] 2024-03-04T17:36:45.584 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-04T17:36:45.611 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-04T17:36:45.651 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T17:36:45.962 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libradosstriper1. 2024-03-04T17:36:45.989 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:36:45.996 INFO:teuthology.orchestra.run.smithi173.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1871-g3aec3903-1jammy [8,625 kB] 2024-03-04T17:36:46.046 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libradosstriper1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:36:46.474 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-common. 2024-03-04T17:36:46.500 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../029-ceph-common_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:36:46.540 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:36:47.812 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-base. 2024-03-04T17:36:47.839 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../030-ceph-base_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:36:47.880 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-base (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:36:48.466 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-04T17:36:48.493 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-04T17:36:48.533 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-04T17:36:48.740 INFO:teuthology.orchestra.run.smithi173.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1871-g3aec3903-1jammy [14.2 kB] 2024-03-04T17:36:48.740 INFO:teuthology.orchestra.run.smithi173.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1871-g3aec3903-1jammy [50.7 MB] 2024-03-04T17:36:48.836 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-cheroot. 2024-03-04T17:36:48.862 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-04T17:36:48.902 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T17:36:49.222 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-04T17:36:49.248 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-04T17:36:49.289 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-04T17:36:49.558 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-04T17:36:49.585 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-04T17:36:49.616 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-04T17:36:49.894 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-04T17:36:49.921 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-04T17:36:49.960 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-04T17:36:50.247 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-tempora. 2024-03-04T17:36:50.273 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-03-04T17:36:50.313 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-04T17:36:50.633 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-portend. 2024-03-04T17:36:50.659 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-03-04T17:36:50.699 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-04T17:36:50.952 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-04T17:36:50.968 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-04T17:36:51.002 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-04T17:36:51.297 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-04T17:36:51.323 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-04T17:36:51.363 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-04T17:36:51.667 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-natsort. 2024-03-04T17:36:51.693 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-03-04T17:36:51.725 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-04T17:36:52.036 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-logutils. 2024-03-04T17:36:52.051 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-03-04T17:36:52.094 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-04T17:36:52.998 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-04T17:36:53.025 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-04T17:36:53.315 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-04T17:36:53.559 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-mako. 2024-03-04T17:36:53.586 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-04T17:36:53.617 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T17:36:53.912 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-04T17:36:53.939 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-04T17:36:53.979 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-04T17:36:54.281 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-04T17:36:54.308 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-04T17:36:54.348 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-04T17:36:54.651 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-webob. 2024-03-04T17:36:54.677 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-04T17:36:54.717 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-04T17:36:55.087 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-waitress. 2024-03-04T17:36:55.114 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-04T17:36:55.157 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T17:36:55.440 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-tempita. 2024-03-04T17:36:55.467 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-04T17:36:55.506 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T17:36:55.809 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-paste. 2024-03-04T17:36:55.836 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-04T17:36:55.875 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T17:36:56.203 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-04T17:36:56.219 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-04T17:36:56.270 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T17:36:56.548 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-04T17:36:56.564 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-04T17:36:56.598 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-04T17:36:56.851 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-webtest. 2024-03-04T17:36:56.867 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-03-04T17:36:56.909 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-04T17:36:57.220 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pecan. 2024-03-04T17:36:57.247 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-04T17:36:57.295 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T17:36:57.650 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-certifi. 2024-03-04T17:36:57.677 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-03-04T17:36:57.716 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-04T17:36:58.053 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-idna. 2024-03-04T17:36:58.080 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-03-04T17:36:58.119 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-04T17:36:58.414 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-urllib3. 2024-03-04T17:36:58.430 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-04T17:36:58.464 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T17:36:58.758 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-requests. 2024-03-04T17:36:58.774 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-04T17:36:58.808 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T17:36:59.086 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-04T17:36:59.113 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-04T17:36:59.144 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T17:36:59.414 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-04T17:36:59.430 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:36:59.464 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:36:59.810 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-04T17:36:59.838 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:36:59.877 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:00.188 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-04T17:37:00.204 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-04T17:37:00.238 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-04T17:37:00.491 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-openssl. 2024-03-04T17:37:00.518 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-03-04T17:37:00.549 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-04T17:37:00.886 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mgr. 2024-03-04T17:37:00.913 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:00.952 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mgr (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:01.355 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mon. 2024-03-04T17:37:01.382 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:01.422 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mon (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:01.967 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-osd. 2024-03-04T17:37:01.993 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:02.025 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-osd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:03.353 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph. 2024-03-04T17:37:03.380 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../066-ceph_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:03.419 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:03.731 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-fuse. 2024-03-04T17:37:03.758 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:03.797 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:04.150 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mds. 2024-03-04T17:37:04.177 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:04.217 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mds (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:04.553 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package cephadm. 2024-03-04T17:37:04.580 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../069-cephadm_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:04.619 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:04.939 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-04T17:37:04.966 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-04T17:37:05.005 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-04T17:37:05.316 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-04T17:37:05.332 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T17:37:05.374 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T17:37:06.415 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-babel. 2024-03-04T17:37:06.443 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T17:37:06.481 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T17:37:06.809 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-jinja2. 2024-03-04T17:37:06.837 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-04T17:37:06.876 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T17:37:07.195 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-04T17:37:07.223 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:37:07.262 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:07.615 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-04T17:37:07.642 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-03-04T17:37:07.682 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-04T17:37:07.943 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-routes. 2024-03-04T17:37:07.969 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-04T17:37:08.001 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T17:37:08.337 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-04T17:37:08.365 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:37:08.404 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:08.874 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-04T17:37:08.890 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-04T17:37:08.932 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T17:37:09.419 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-joblib. 2024-03-04T17:37:09.446 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-04T17:37:09.485 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T17:37:09.864 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-04T17:37:09.891 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-04T17:37:09.930 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-04T17:37:10.241 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-sklearn. 2024-03-04T17:37:10.269 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-04T17:37:10.308 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T17:37:10.991 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-04T17:37:11.007 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:37:11.127 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:12.088 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-cachetools. 2024-03-04T17:37:12.115 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-03-04T17:37:12.155 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-04T17:37:12.441 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-04T17:37:12.457 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-04T17:37:12.491 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-04T17:37:12.803 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-04T17:37:12.830 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-04T17:37:12.861 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T17:37:13.147 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-rsa. 2024-03-04T17:37:13.174 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-03-04T17:37:13.213 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-04T17:37:13.709 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-google-auth. 2024-03-04T17:37:13.736 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-03-04T17:37:13.775 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-04T17:37:14.204 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-04T17:37:14.231 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-04T17:37:14.261 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T17:37:14.548 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-websocket. 2024-03-04T17:37:14.575 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-03-04T17:37:14.614 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-04T17:37:14.926 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-04T17:37:14.953 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-04T17:37:14.992 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T17:37:15.673 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-04T17:37:15.701 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:37:15.739 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:16.059 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-04T17:37:16.087 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-04T17:37:16.125 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T17:37:16.470 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-04T17:37:16.498 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T17:37:16.536 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T17:37:16.756 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package jq. 2024-03-04T17:37:16.772 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T17:37:16.807 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-04T17:37:17.050 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package socat. 2024-03-04T17:37:17.077 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-04T17:37:17.120 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-04T17:37:17.429 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package xmlstarlet. 2024-03-04T17:37:17.456 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-04T17:37:17.495 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-04T17:37:17.858 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-test. 2024-03-04T17:37:17.873 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../097-ceph-test_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:17.907 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-test (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:20.178 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-volume. 2024-03-04T17:37:20.206 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:37:20.236 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-volume (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:20.548 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-04T17:37:20.576 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:20.615 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libcephfs-dev (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:20.960 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-04T17:37:20.987 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-04T17:37:21.026 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T17:37:21.339 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-04T17:37:21.366 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-03-04T17:37:21.405 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-04T17:37:21.718 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package nvme-cli. 2024-03-04T17:37:21.745 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-04T17:37:21.784 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T17:37:22.164 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package pkg-config. 2024-03-04T17:37:22.192 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-04T17:37:22.230 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T17:37:22.475 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-04T17:37:22.502 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-04T17:37:22.543 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-04T17:37:22.911 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-04T17:37:22.927 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-04T17:37:22.961 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-04T17:37:23.264 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pastescript. 2024-03-04T17:37:23.292 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-03-04T17:37:23.330 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-04T17:37:23.650 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pluggy. 2024-03-04T17:37:23.678 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-04T17:37:23.717 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-04T17:37:24.028 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-psutil. 2024-03-04T17:37:24.055 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-04T17:37:24.119 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-04T17:37:24.456 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-py. 2024-03-04T17:37:24.483 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-03-04T17:37:24.530 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-04T17:37:24.859 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pygments. 2024-03-04T17:37:24.887 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-04T17:37:24.925 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T17:37:25.371 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-04T17:37:25.387 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-04T17:37:25.421 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-04T17:37:25.716 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-toml. 2024-03-04T17:37:25.742 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-03-04T17:37:25.781 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-04T17:37:26.068 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pytest. 2024-03-04T17:37:26.095 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-04T17:37:26.134 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T17:37:26.488 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-simplejson. 2024-03-04T17:37:26.515 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-04T17:37:26.554 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-04T17:37:26.832 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-04T17:37:26.859 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-04T17:37:26.898 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-04T17:37:27.586 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package radosgw. 2024-03-04T17:37:27.613 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../116-radosgw_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:27.644 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking radosgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:28.422 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package rbd-fuse. 2024-03-04T17:37:28.446 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:28.488 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking rbd-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:28.615 INFO:teuthology.orchestra.run.smithi173.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1871-g3aec3903-1jammy [121 kB] 2024-03-04T17:37:28.711 INFO:teuthology.orchestra.run.smithi173.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1871-g3aec3903-1jammy [38.4 kB] 2024-03-04T17:37:28.745 INFO:teuthology.orchestra.run.smithi173.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1871-g3aec3903-1jammy [13.1 MB] 2024-03-04T17:37:28.793 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-04T17:37:29.069 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T17:37:29.195 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T17:37:29.431 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-04T17:37:29.767 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-04T17:37:30.157 INFO:teuthology.orchestra.run.smithi137.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-04T17:37:30.221 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:30.755 INFO:teuthology.orchestra.run.smithi137.stdout:Adding system user cephadm....done 2024-03-04T17:37:30.910 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T17:37:31.195 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-04T17:37:31.446 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-04T17:37:31.572 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-04T17:37:31.806 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-04T17:37:32.058 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T17:37:32.184 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-04T17:37:32.485 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T17:37:32.846 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-04T17:37:33.080 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-04T17:37:33.340 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-04T17:37:33.567 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-04T17:37:33.817 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-04T17:37:34.077 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-04T17:37:34.336 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-ceph-argparse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:34.587 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T17:37:34.688 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T17:37:34.789 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T17:37:34.932 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-04T17:37:35.200 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-04T17:37:35.583 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-04T17:37:35.614 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-04T17:37:35.669 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T17:37:35.787 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-04T17:37:36.146 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T17:37:36.247 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-04T17:37:36.507 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T17:37:36.758 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-04T17:37:37.067 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up zip (3.0-12build2) ... 2024-03-04T17:37:37.193 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T17:37:37.854 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T17:37:38.113 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T17:37:38.214 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-04T17:37:38.315 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T17:37:38.510 INFO:teuthology.orchestra.run.smithi173.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-rgw-d4n/3aec390336fdfdc8f927818b718cfb0361bd47c8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1871-g3aec3903-1jammy [91.7 kB] 2024-03-04T17:37:38.599 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-04T17:37:38.976 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-04T17:37:39.117 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 177 MB in 1min 18s (2,260 kB/s) 2024-03-04T17:37:39.218 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T17:37:39.257 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-04T17:37:39.355 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165994 files and directories currently installed.) 2024-03-04T17:37:39.361 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-04T17:37:39.430 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T17:37:39.595 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T17:37:39.863 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T17:37:39.875 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-04T17:37:39.901 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-04T17:37:39.933 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T17:37:40.189 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-04T17:37:40.278 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-04T17:37:40.304 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-04T17:37:40.344 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T17:37:40.432 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-04T17:37:40.558 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:40.790 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-04T17:37:40.816 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T17:37:40.859 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-idna (3.3-1) ... 2024-03-04T17:37:40.863 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:37:41.177 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T17:37:41.217 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-04T17:37:41.233 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T17:37:41.267 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:37:41.662 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-04T17:37:41.689 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T17:37:41.737 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:37:42.115 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-04T17:37:42.142 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-04T17:37:42.182 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T17:37:42.308 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T17:37:42.501 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librados2. 2024-03-04T17:37:42.528 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../007-librados2_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:42.551 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:37:42.568 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librados2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:42.685 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-04T17:37:42.961 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T17:37:43.062 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-04T17:37:43.079 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librbd1. 2024-03-04T17:37:43.105 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../008-librbd1_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:43.154 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librbd1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:43.163 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T17:37:43.431 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-04T17:37:43.648 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libcephfs2. 2024-03-04T17:37:43.674 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:43.691 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-04T17:37:43.723 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libcephfs2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:43.941 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:37:44.001 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rados. 2024-03-04T17:37:44.016 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../010-python3-rados_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:44.059 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rados (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:44.067 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-04T17:37:44.369 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-04T17:37:44.378 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-04T17:37:44.405 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:37:44.445 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-ceph-argparse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:44.678 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-04T17:37:44.739 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cephfs. 2024-03-04T17:37:44.755 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:44.806 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cephfs (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:44.921 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-04T17:37:45.155 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-04T17:37:45.175 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-04T17:37:45.202 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:37:45.275 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:45.398 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-04T17:37:45.628 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-04T17:37:45.654 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-04T17:37:45.657 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T17:37:45.695 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T17:37:45.783 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-04T17:37:46.014 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-prettytable. 2024-03-04T17:37:46.040 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-03-04T17:37:46.059 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T17:37:46.081 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-04T17:37:46.185 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T17:37:46.325 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rbd. 2024-03-04T17:37:46.340 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:46.391 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rbd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:46.470 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-04T17:37:46.754 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-04T17:37:46.794 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-04T17:37:46.820 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-04T17:37:46.872 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T17:37:46.989 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T17:37:47.115 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-04T17:37:47.180 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-04T17:37:47.207 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-04T17:37:47.241 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T17:37:47.247 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T17:37:47.330 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-04T17:37:47.550 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-04T17:37:47.559 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T17:37:47.576 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-04T17:37:47.616 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T17:37:47.687 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T17:37:47.961 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package lua5.1. 2024-03-04T17:37:47.988 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-04T17:37:48.036 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-04T17:37:48.078 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-04T17:37:48.329 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-04T17:37:48.397 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package lua-any. 2024-03-04T17:37:48.424 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-03-04T17:37:48.430 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-04T17:37:48.472 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-04T17:37:48.689 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:37:48.733 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package zip. 2024-03-04T17:37:48.759 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-03-04T17:37:48.800 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking zip (3.0-12build2) ... 2024-03-04T17:37:48.816 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T17:37:49.133 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T17:37:49.153 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package unzip. 2024-03-04T17:37:49.179 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-04T17:37:49.220 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-04T17:37:49.459 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T17:37:49.682 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package luarocks. 2024-03-04T17:37:49.709 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-04T17:37:49.735 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-04T17:37:49.757 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-04T17:37:49.861 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-04T17:37:50.129 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-04T17:37:50.198 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librgw2. 2024-03-04T17:37:50.225 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../025-librgw2_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:50.264 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librgw2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:50.380 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T17:37:50.727 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-04T17:37:50.808 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rgw. 2024-03-04T17:37:50.823 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:50.866 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:51.095 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-04T17:37:51.202 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-04T17:37:51.229 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-04T17:37:51.269 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T17:37:51.363 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T17:37:51.647 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libradosstriper1. 2024-03-04T17:37:51.673 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:51.674 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T17:37:51.722 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libradosstriper1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:51.809 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up librados2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:51.935 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:52.061 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-04T17:37:52.167 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-common. 2024-03-04T17:37:52.194 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../029-ceph-common_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:52.233 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:52.339 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libcephfs2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:52.465 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libradosstriper1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:52.591 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-04T17:37:52.867 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up librbd1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:52.993 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mgr-modules-core (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:53.119 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:53.315 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T17:37:53.315 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T17:37:53.589 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-base. 2024-03-04T17:37:53.615 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../030-ceph-base_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:37:53.664 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-base (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:53.815 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libcephfs-dev (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:53.913 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-rados (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:54.014 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up librgw2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:54.123 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T17:37:54.251 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-04T17:37:54.277 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-04T17:37:54.317 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-04T17:37:54.629 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cheroot. 2024-03-04T17:37:54.655 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-04T17:37:54.695 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T17:37:55.015 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-04T17:37:55.041 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-04T17:37:55.081 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-04T17:37:55.294 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-rbd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:55.384 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-04T17:37:55.411 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-04T17:37:55.411 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up rbd-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:55.451 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-04T17:37:55.512 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-rgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:55.613 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-cephfs (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:55.720 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-04T17:37:55.739 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:55.747 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-04T17:37:55.778 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-04T17:37:56.056 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-tempora. 2024-03-04T17:37:56.083 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-03-04T17:37:56.123 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-04T17:37:56.266 INFO:teuthology.orchestra.run.smithi137.stdout:Adding group ceph....done 2024-03-04T17:37:56.426 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-portend. 2024-03-04T17:37:56.452 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-03-04T17:37:56.492 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-04T17:37:56.793 INFO:teuthology.orchestra.run.smithi137.stdout:Adding system user ceph....done 2024-03-04T17:37:56.820 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-04T17:37:56.846 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-04T17:37:56.887 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-04T17:37:56.960 INFO:teuthology.orchestra.run.smithi137.stdout:Setting system user ceph properties....done 2024-03-04T17:37:56.979 INFO:teuthology.orchestra.run.smithi137.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-04T17:37:57.124 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-04T17:37:57.148 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-04T17:37:57.174 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-04T17:37:57.206 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-04T17:37:57.455 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-04T17:37:57.543 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-natsort. 2024-03-04T17:37:57.569 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-03-04T17:37:57.609 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-04T17:37:57.921 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-logutils. 2024-03-04T17:37:57.923 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-test (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:57.947 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-03-04T17:37:57.987 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-04T17:37:58.049 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up radosgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:58.256 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-04T17:37:58.272 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-04T17:37:58.307 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-04T17:37:58.525 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T17:37:58.525 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T17:37:58.593 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-mako. 2024-03-04T17:37:58.619 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-04T17:37:58.659 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T17:37:58.951 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-base (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:58.963 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-04T17:37:58.989 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-04T17:37:59.029 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-04T17:37:59.301 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-04T17:37:59.315 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-04T17:37:59.342 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-04T17:37:59.381 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-04T17:37:59.659 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-webob. 2024-03-04T17:37:59.686 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-04T17:37:59.720 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mds (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:37:59.726 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-04T17:37:59.934 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T17:37:59.934 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T17:38:00.029 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-waitress. 2024-03-04T17:38:00.055 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-04T17:38:00.090 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T17:38:00.373 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-tempita. 2024-03-04T17:38:00.399 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-04T17:38:00.413 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mgr (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:00.440 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T17:38:00.626 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T17:38:00.626 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T17:38:00.776 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-paste. 2024-03-04T17:38:00.803 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-04T17:38:00.851 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T17:38:01.081 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-osd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:01.171 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-04T17:38:01.186 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-04T17:38:01.221 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T17:38:01.360 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T17:38:01.360 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T17:38:01.516 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-04T17:38:01.542 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-04T17:38:01.582 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-04T17:38:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:01.885 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-webtest. 2024-03-04T17:38:01.911 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-03-04T17:38:01.942 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:01.951 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-04T17:38:02.093 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mon (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:02.279 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pecan. 2024-03-04T17:38:02.305 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T17:38:02.305 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T17:38:02.306 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-04T17:38:02.345 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T17:38:02.700 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-certifi. 2024-03-04T17:38:02.727 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-03-04T17:38:02.767 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-04T17:38:02.777 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mgr-cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:02.912 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:03.016 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mgr-dashboard (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:03.128 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-idna. 2024-03-04T17:38:03.147 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-volume (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:03.155 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-03-04T17:38:03.203 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-04T17:38:03.491 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-04T17:38:03.573 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-urllib3. 2024-03-04T17:38:03.599 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-04T17:38:03.643 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-04T17:38:03.647 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T17:38:03.787 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T17:38:04.250 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-requests. 2024-03-04T17:38:04.277 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-04T17:38:04.325 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T17:38:04.695 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-04T17:38:04.721 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-04T17:38:04.761 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T17:38:04.904 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-04T17:38:05.073 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-04T17:38:05.100 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:38:05.139 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:05.544 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-04T17:38:05.571 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:05.606 INFO:teuthology.orchestra.run.smithi137.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-04T17:38:05.609 DEBUG:teuthology.orchestra.run.smithi137:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install s3cmd maven 2024-03-04T17:38:05.610 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:05.663 INFO:teuthology.orchestra.run.smithi137.stdout:Reading package lists... 2024-03-04T17:38:05.858 INFO:teuthology.orchestra.run.smithi137.stdout:Building dependency tree... 2024-03-04T17:38:05.859 INFO:teuthology.orchestra.run.smithi137.stdout:Reading state information... 2024-03-04T17:38:05.930 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-04T17:38:05.957 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-04T17:38:05.996 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-04T17:38:06.020 INFO:teuthology.orchestra.run.smithi137.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:38:06.020 INFO:teuthology.orchestra.run.smithi137.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:38:06.020 INFO:teuthology.orchestra.run.smithi137.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:38:06.020 INFO:teuthology.orchestra.run.smithi137.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:38:06.021 INFO:teuthology.orchestra.run.smithi137.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T17:38:06.021 INFO:teuthology.orchestra.run.smithi137.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T17:38:06.021 INFO:teuthology.orchestra.run.smithi137.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:38:06.022 INFO:teuthology.orchestra.run.smithi137.stdout:The following additional packages will be installed: 2024-03-04T17:38:06.022 INFO:teuthology.orchestra.run.smithi137.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-03-04T17:38:06.022 INFO:teuthology.orchestra.run.smithi137.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-03-04T17:38:06.022 INFO:teuthology.orchestra.run.smithi137.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-03-04T17:38:06.022 INFO:teuthology.orchestra.run.smithi137.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-03-04T17:38:06.022 INFO:teuthology.orchestra.run.smithi137.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-03-04T17:38:06.022 INFO:teuthology.orchestra.run.smithi137.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-03-04T17:38:06.022 INFO:teuthology.orchestra.run.smithi137.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-03-04T17:38:06.022 INFO:teuthology.orchestra.run.smithi137.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-03-04T17:38:06.023 INFO:teuthology.orchestra.run.smithi137.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-03-04T17:38:06.023 INFO:teuthology.orchestra.run.smithi137.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-03-04T17:38:06.023 INFO:teuthology.orchestra.run.smithi137.stdout: libwagon-http-shaded-java libwagon-provider-api-java python3-magic 2024-03-04T17:38:06.023 INFO:teuthology.orchestra.run.smithi137.stdout:Suggested packages: 2024-03-04T17:38:06.023 INFO:teuthology.orchestra.run.smithi137.stdout: libaopalliance-java-doc libatinject-jsr330-api-java-doc libel-api-java 2024-03-04T17:38:06.023 INFO:teuthology.orchestra.run.smithi137.stdout: libcommons-io-java-doc libcommons-lang3-java-doc libasm-java libcglib-java 2024-03-04T17:38:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: libjsr305-java-doc libmaven-shared-utils-java-doc liblogback-java 2024-03-04T17:38:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: libplexus-classworlds-java-doc libplexus-sec-dispatcher-java-doc 2024-03-04T17:38:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: libplexus-utils2-java-doc testng libcommons-logging-java liblog4j1.2-java 2024-03-04T17:38:06.060 INFO:teuthology.orchestra.run.smithi137.stdout:The following NEW packages will be installed: 2024-03-04T17:38:06.060 INFO:teuthology.orchestra.run.smithi137.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-03-04T17:38:06.060 INFO:teuthology.orchestra.run.smithi137.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-03-04T17:38:06.061 INFO:teuthology.orchestra.run.smithi137.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-03-04T17:38:06.061 INFO:teuthology.orchestra.run.smithi137.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-03-04T17:38:06.061 INFO:teuthology.orchestra.run.smithi137.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-03-04T17:38:06.061 INFO:teuthology.orchestra.run.smithi137.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-03-04T17:38:06.061 INFO:teuthology.orchestra.run.smithi137.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-03-04T17:38:06.061 INFO:teuthology.orchestra.run.smithi137.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-03-04T17:38:06.061 INFO:teuthology.orchestra.run.smithi137.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-03-04T17:38:06.061 INFO:teuthology.orchestra.run.smithi137.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-03-04T17:38:06.062 INFO:teuthology.orchestra.run.smithi137.stdout: libwagon-http-shaded-java libwagon-provider-api-java maven python3-magic 2024-03-04T17:38:06.062 INFO:teuthology.orchestra.run.smithi137.stdout: s3cmd 2024-03-04T17:38:06.142 INFO:teuthology.orchestra.run.smithi137.stdout:0 upgraded, 35 newly installed, 0 to remove and 275 not upgraded. 2024-03-04T17:38:06.143 INFO:teuthology.orchestra.run.smithi137.stdout:Need to get 10.4 MB of archives. 2024-03-04T17:38:06.143 INFO:teuthology.orchestra.run.smithi137.stdout:After this operation, 13.9 MB of additional disk space will be used. 2024-03-04T17:38:06.143 INFO:teuthology.orchestra.run.smithi137.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libapache-pom-java all 18-1 [4,720 B] 2024-03-04T17:38:06.174 INFO:teuthology.orchestra.run.smithi137.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libatinject-jsr330-api-java all 1.0+ds1-5 [5,348 B] 2024-03-04T17:38:06.174 INFO:teuthology.orchestra.run.smithi137.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-interceptor-3.0-spec-java all 1.0.1-4fakesync [8,616 B] 2024-03-04T17:38:06.183 INFO:teuthology.orchestra.run.smithi137.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcdi-api-java all 1.2-3 [54.3 kB] 2024-03-04T17:38:06.249 INFO:teuthology.orchestra.run.smithi137.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-cli-java all 1.4-2 [55.8 kB] 2024-03-04T17:38:06.274 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-openssl. 2024-03-04T17:38:06.278 INFO:teuthology.orchestra.run.smithi137.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-parent-java all 43-1 [10.8 kB] 2024-03-04T17:38:06.280 INFO:teuthology.orchestra.run.smithi137.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-io-java all 2.11.0-2 [297 kB] 2024-03-04T17:38:06.301 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-03-04T17:38:06.339 INFO:teuthology.orchestra.run.smithi137.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-lang3-java all 3.11-1 [526 kB] 2024-03-04T17:38:06.341 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-04T17:38:06.382 INFO:teuthology.orchestra.run.smithi137.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-annotation-1.3-spec-java all 1.3-1 [11.2 kB] 2024-03-04T17:38:06.383 INFO:teuthology.orchestra.run.smithi137.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjsr305-java all 0.1~+svn49-11 [27.0 kB] 2024-03-04T17:38:06.384 INFO:teuthology.orchestra.run.smithi137.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguava-java all 29.0-6 [2,418 kB] 2024-03-04T17:38:06.463 INFO:teuthology.orchestra.run.smithi137.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libaopalliance-java all 20070526-6 [9,084 B] 2024-03-04T17:38:06.463 INFO:teuthology.orchestra.run.smithi137.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguice-java all 4.2.3-2 [1,434 kB] 2024-03-04T17:38:06.490 INFO:teuthology.orchestra.run.smithi137.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libhawtjni-runtime-java all 1.17-1 [28.8 kB] 2024-03-04T17:38:06.491 INFO:teuthology.orchestra.run.smithi137.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-native-java all 1.8-1 [23.8 kB] 2024-03-04T17:38:06.491 INFO:teuthology.orchestra.run.smithi137.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-java all 1.18-1 [56.8 kB] 2024-03-04T17:38:06.493 INFO:teuthology.orchestra.run.smithi137.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-parent-java all 31-2 [5,140 B] 2024-03-04T17:38:06.493 INFO:teuthology.orchestra.run.smithi137.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-utils2-java all 3.3.0-1 [250 kB] 2024-03-04T17:38:06.497 INFO:teuthology.orchestra.run.smithi137.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-provider-api-java all 3.3.4-1 [48.5 kB] 2024-03-04T17:38:06.497 INFO:teuthology.orchestra.run.smithi137.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-resolver-java all 1.4.2-3build1 [555 kB] 2024-03-04T17:38:06.521 INFO:teuthology.orchestra.run.smithi137.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-shared-utils-java all 3.3.0-1 [149 kB] 2024-03-04T17:38:06.523 INFO:teuthology.orchestra.run.smithi137.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-cipher-java all 1.8-2 [15.1 kB] 2024-03-04T17:38:06.546 INFO:teuthology.orchestra.run.smithi137.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-classworlds-java all 2.6.0-1 [49.4 kB] 2024-03-04T17:38:06.547 INFO:teuthology.orchestra.run.smithi137.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-component-annotations-java all 2.1.0-1 [6,564 B] 2024-03-04T17:38:06.547 INFO:teuthology.orchestra.run.smithi137.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-interpolation-java all 1.26-1 [76.8 kB] 2024-03-04T17:38:06.548 INFO:teuthology.orchestra.run.smithi137.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-sec-dispatcher-java all 1.4-4 [28.1 kB] 2024-03-04T17:38:06.549 INFO:teuthology.orchestra.run.smithi137.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libslf4j-java all 1.7.32-1 [141 kB] 2024-03-04T17:38:06.551 INFO:teuthology.orchestra.run.smithi137.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-inject-java all 0.3.4-2 [347 kB] 2024-03-04T17:38:06.555 INFO:teuthology.orchestra.run.smithi137.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-plexus-java all 0.3.4-3 [181 kB] 2024-03-04T17:38:06.563 INFO:teuthology.orchestra.run.smithi137.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven3-core-java all 3.6.3-5 [1,535 kB] 2024-03-04T17:38:06.605 INFO:teuthology.orchestra.run.smithi137.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-file-java all 3.3.4-1 [7,928 B] 2024-03-04T17:38:06.605 INFO:teuthology.orchestra.run.smithi137.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-http-shaded-java all 3.3.4-1 [1,855 kB] 2024-03-04T17:38:06.629 INFO:teuthology.orchestra.run.smithi137.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 maven all 3.6.3-5 [17.8 kB] 2024-03-04T17:38:06.629 INFO:teuthology.orchestra.run.smithi137.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-magic all 2:0.4.24-2 [12.6 kB] 2024-03-04T17:38:06.629 INFO:teuthology.orchestra.run.smithi137.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/universe amd64 s3cmd all 2.2.0-1 [120 kB] 2024-03-04T17:38:06.652 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr. 2024-03-04T17:38:06.679 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:06.719 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:07.070 INFO:teuthology.orchestra.run.smithi137.stdout:Fetched 10.4 MB in 1s (18.5 MB/s) 2024-03-04T17:38:07.122 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mon. 2024-03-04T17:38:07.149 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:07.168 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libapache-pom-java. 2024-03-04T17:38:07.188 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mon (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:07.229 INFO:teuthology.orchestra.run.smithi137.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173563 files and directories currently installed.) 2024-03-04T17:38:07.234 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../00-libapache-pom-java_18-1_all.deb ... 2024-03-04T17:38:07.276 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libapache-pom-java (18-1) ... 2024-03-04T17:38:07.528 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libatinject-jsr330-api-java. 2024-03-04T17:38:07.544 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../01-libatinject-jsr330-api-java_1.0+ds1-5_all.deb ... 2024-03-04T17:38:07.578 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-03-04T17:38:07.772 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libgeronimo-interceptor-3.0-spec-java. 2024-03-04T17:38:07.788 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../02-libgeronimo-interceptor-3.0-spec-java_1.0.1-4fakesync_all.deb ... 2024-03-04T17:38:07.809 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-osd. 2024-03-04T17:38:07.831 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-03-04T17:38:07.835 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:07.875 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-osd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:08.083 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libcdi-api-java. 2024-03-04T17:38:08.099 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../03-libcdi-api-java_1.2-3_all.deb ... 2024-03-04T17:38:08.150 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libcdi-api-java (1.2-3) ... 2024-03-04T17:38:08.411 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libcommons-cli-java. 2024-03-04T17:38:08.438 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../04-libcommons-cli-java_1.4-2_all.deb ... 2024-03-04T17:38:08.477 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libcommons-cli-java (1.4-2) ... 2024-03-04T17:38:08.721 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libcommons-parent-java. 2024-03-04T17:38:08.737 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../05-libcommons-parent-java_43-1_all.deb ... 2024-03-04T17:38:08.771 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libcommons-parent-java (43-1) ... 2024-03-04T17:38:08.911 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph. 2024-03-04T17:38:08.927 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../066-ceph_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:08.969 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:08.982 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libcommons-io-java. 2024-03-04T17:38:08.998 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../06-libcommons-io-java_2.11.0-2_all.deb ... 2024-03-04T17:38:09.032 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libcommons-io-java (2.11.0-2) ... 2024-03-04T17:38:09.298 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-fuse. 2024-03-04T17:38:09.310 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libcommons-lang3-java. 2024-03-04T17:38:09.324 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:09.326 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../07-libcommons-lang3-java_3.11-1_all.deb ... 2024-03-04T17:38:09.364 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:09.368 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libcommons-lang3-java (3.11-1) ... 2024-03-04T17:38:09.670 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libgeronimo-annotation-1.3-spec-java. 2024-03-04T17:38:09.698 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../08-libgeronimo-annotation-1.3-spec-java_1.3-1_all.deb ... 2024-03-04T17:38:09.737 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-03-04T17:38:09.742 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mds. 2024-03-04T17:38:09.769 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:09.808 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mds (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:09.981 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libjsr305-java. 2024-03-04T17:38:09.997 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../09-libjsr305-java_0.1~+svn49-11_all.deb ... 2024-03-04T17:38:10.030 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libjsr305-java (0.1~+svn49-11) ... 2024-03-04T17:38:10.170 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package cephadm. 2024-03-04T17:38:10.186 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../069-cephadm_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:10.219 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:10.344 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libguava-java. 2024-03-04T17:38:10.371 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../10-libguava-java_29.0-6_all.deb ... 2024-03-04T17:38:10.427 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libguava-java (29.0-6) ... 2024-03-04T17:38:10.533 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-04T17:38:10.560 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-04T17:38:10.599 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-04T17:38:10.871 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libaopalliance-java. 2024-03-04T17:38:10.887 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../11-libaopalliance-java_20070526-6_all.deb ... 2024-03-04T17:38:10.901 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-04T17:38:10.917 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T17:38:10.929 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libaopalliance-java (20070526-6) ... 2024-03-04T17:38:10.960 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T17:38:11.181 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libguice-java. 2024-03-04T17:38:11.197 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../12-libguice-java_4.2.3-2_all.deb ... 2024-03-04T17:38:11.231 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libguice-java (4.2.3-2) ... 2024-03-04T17:38:11.585 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libhawtjni-runtime-java. 2024-03-04T17:38:11.601 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../13-libhawtjni-runtime-java_1.17-1_all.deb ... 2024-03-04T17:38:11.644 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libhawtjni-runtime-java (1.17-1) ... 2024-03-04T17:38:11.896 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libjansi-native-java. 2024-03-04T17:38:11.912 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../14-libjansi-native-java_1.8-1_all.deb ... 2024-03-04T17:38:11.954 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libjansi-native-java (1.8-1) ... 2024-03-04T17:38:12.015 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-babel. 2024-03-04T17:38:12.042 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T17:38:12.081 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T17:38:12.198 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libjansi-java. 2024-03-04T17:38:12.215 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../15-libjansi-java_1.18-1_all.deb ... 2024-03-04T17:38:12.248 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libjansi-java (1.18-1) ... 2024-03-04T17:38:12.426 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jinja2. 2024-03-04T17:38:12.451 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libmaven-parent-java. 2024-03-04T17:38:12.453 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-04T17:38:12.467 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../16-libmaven-parent-java_31-2_all.deb ... 2024-03-04T17:38:12.492 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T17:38:12.501 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libmaven-parent-java (31-2) ... 2024-03-04T17:38:12.753 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libplexus-utils2-java. 2024-03-04T17:38:12.769 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../17-libplexus-utils2-java_3.3.0-1_all.deb ... 2024-03-04T17:38:12.811 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libplexus-utils2-java (3.3.0-1) ... 2024-03-04T17:38:12.820 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-04T17:38:12.847 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:38:12.887 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:13.089 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libwagon-provider-api-java. 2024-03-04T17:38:13.116 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../18-libwagon-provider-api-java_3.3.4-1_all.deb ... 2024-03-04T17:38:13.156 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libwagon-provider-api-java (3.3.4-1) ... 2024-03-04T17:38:13.232 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-04T17:38:13.259 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-03-04T17:38:13.298 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-04T17:38:13.400 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libmaven-resolver-java. 2024-03-04T17:38:13.416 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../19-libmaven-resolver-java_1.4.2-3build1_all.deb ... 2024-03-04T17:38:13.450 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libmaven-resolver-java (1.4.2-3build1) ... 2024-03-04T17:38:13.601 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-routes. 2024-03-04T17:38:13.628 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-04T17:38:13.667 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T17:38:13.746 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libmaven-shared-utils-java. 2024-03-04T17:38:13.793 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../20-libmaven-shared-utils-java_3.3.0-1_all.deb ... 2024-03-04T17:38:13.812 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libmaven-shared-utils-java (3.3.0-1) ... 2024-03-04T17:38:14.038 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-04T17:38:14.064 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:38:14.081 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libplexus-cipher-java. 2024-03-04T17:38:14.097 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../21-libplexus-cipher-java_1.8-2_all.deb ... 2024-03-04T17:38:14.104 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:14.139 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libplexus-cipher-java (1.8-2) ... 2024-03-04T17:38:14.417 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libplexus-classworlds-java. 2024-03-04T17:38:14.445 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../22-libplexus-classworlds-java_2.6.0-1_all.deb ... 2024-03-04T17:38:14.492 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libplexus-classworlds-java (2.6.0-1) ... 2024-03-04T17:38:14.508 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-04T17:38:14.523 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-04T17:38:14.566 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T17:38:14.736 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libplexus-component-annotations-java. 2024-03-04T17:38:14.752 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../23-libplexus-component-annotations-java_2.1.0-1_all.deb ... 2024-03-04T17:38:14.786 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libplexus-component-annotations-java (2.1.0-1) ... 2024-03-04T17:38:14.997 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libplexus-interpolation-java. 2024-03-04T17:38:15.013 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../24-libplexus-interpolation-java_1.26-1_all.deb ... 2024-03-04T17:38:15.055 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libplexus-interpolation-java (1.26-1) ... 2024-03-04T17:38:15.077 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-joblib. 2024-03-04T17:38:15.104 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-04T17:38:15.144 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T17:38:15.409 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libplexus-sec-dispatcher-java. 2024-03-04T17:38:15.437 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../25-libplexus-sec-dispatcher-java_1.4-4_all.deb ... 2024-03-04T17:38:15.475 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libplexus-sec-dispatcher-java (1.4-4) ... 2024-03-04T17:38:15.497 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-04T17:38:15.513 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-04T17:38:15.547 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-04T17:38:15.728 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libslf4j-java. 2024-03-04T17:38:15.756 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../26-libslf4j-java_1.7.32-1_all.deb ... 2024-03-04T17:38:15.795 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libslf4j-java (1.7.32-1) ... 2024-03-04T17:38:15.808 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-sklearn. 2024-03-04T17:38:15.835 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-04T17:38:15.866 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T17:38:16.031 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libsisu-inject-java. 2024-03-04T17:38:16.047 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../27-libsisu-inject-java_0.3.4-2_all.deb ... 2024-03-04T17:38:16.081 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libsisu-inject-java (0.3.4-2) ... 2024-03-04T17:38:16.376 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libsisu-plexus-java. 2024-03-04T17:38:16.404 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../28-libsisu-plexus-java_0.3.4-3_all.deb ... 2024-03-04T17:38:16.442 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libsisu-plexus-java (0.3.4-3) ... 2024-03-04T17:38:16.575 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-04T17:38:16.602 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:38:16.720 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libmaven3-core-java. 2024-03-04T17:38:16.735 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:16.736 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../29-libmaven3-core-java_3.6.3-5_all.deb ... 2024-03-04T17:38:16.778 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libmaven3-core-java (3.6.3-5) ... 2024-03-04T17:38:17.157 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libwagon-file-java. 2024-03-04T17:38:17.173 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../30-libwagon-file-java_3.3.4-1_all.deb ... 2024-03-04T17:38:17.216 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libwagon-file-java (3.3.4-1) ... 2024-03-04T17:38:17.435 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libwagon-http-shaded-java. 2024-03-04T17:38:17.451 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../31-libwagon-http-shaded-java_3.3.4-1_all.deb ... 2024-03-04T17:38:17.485 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libwagon-http-shaded-java (3.3.4-1) ... 2024-03-04T17:38:17.639 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cachetools. 2024-03-04T17:38:17.666 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-03-04T17:38:17.705 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-04T17:38:17.963 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package maven. 2024-03-04T17:38:17.990 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../32-maven_3.6.3-5_all.deb ... 2024-03-04T17:38:18.008 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-04T17:38:18.029 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking maven (3.6.3-5) ... 2024-03-04T17:38:18.035 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-04T17:38:18.074 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-04T17:38:18.341 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-magic. 2024-03-04T17:38:18.368 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../33-python3-magic_2%3a0.4.24-2_all.deb ... 2024-03-04T17:38:18.395 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-04T17:38:18.407 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-magic (2:0.4.24-2) ... 2024-03-04T17:38:18.422 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-04T17:38:18.461 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T17:38:18.710 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package s3cmd. 2024-03-04T17:38:18.738 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../34-s3cmd_2.2.0-1_all.deb ... 2024-03-04T17:38:18.772 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rsa. 2024-03-04T17:38:18.776 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking s3cmd (2.2.0-1) ... 2024-03-04T17:38:18.800 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-03-04T17:38:18.839 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-04T17:38:19.126 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-google-auth. 2024-03-04T17:38:19.146 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libslf4j-java (1.7.32-1) ... 2024-03-04T17:38:19.153 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-03-04T17:38:19.184 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-04T17:38:19.273 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libplexus-utils2-java (3.3.0-1) ... 2024-03-04T17:38:19.382 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libplexus-classworlds-java (2.6.0-1) ... 2024-03-04T17:38:19.437 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-04T17:38:19.453 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-04T17:38:19.483 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libjsr305-java (0.1~+svn49-11) ... 2024-03-04T17:38:19.487 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T17:38:19.601 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libaopalliance-java (20070526-6) ... 2024-03-04T17:38:19.727 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libcommons-cli-java (1.4-2) ... 2024-03-04T17:38:19.823 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-websocket. 2024-03-04T17:38:19.851 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-03-04T17:38:19.853 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libplexus-component-annotations-java (2.1.0-1) ... 2024-03-04T17:38:19.890 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-04T17:38:19.979 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libplexus-cipher-java (1.8-2) ... 2024-03-04T17:38:20.105 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-03-04T17:38:20.202 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-04T17:38:20.229 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-04T17:38:20.231 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-03-04T17:38:20.268 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T17:38:20.357 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libapache-pom-java (18-1) ... 2024-03-04T17:38:20.517 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-03-04T17:38:20.618 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libplexus-interpolation-java (1.26-1) ... 2024-03-04T17:38:20.744 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libplexus-sec-dispatcher-java (1.4-4) ... 2024-03-04T17:38:20.870 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libwagon-http-shaded-java (3.3.4-1) ... 2024-03-04T17:38:20.990 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-04T17:38:20.996 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-magic (2:0.4.24-2) ... 2024-03-04T17:38:21.006 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:38:21.048 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:21.255 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libcdi-api-java (1.2-3) ... 2024-03-04T17:38:21.376 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-04T17:38:21.381 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libhawtjni-runtime-java (1.17-1) ... 2024-03-04T17:38:21.403 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-04T17:38:21.442 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T17:38:21.532 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libwagon-provider-api-java (3.3.4-1) ... 2024-03-04T17:38:21.658 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libmaven-parent-java (31-2) ... 2024-03-04T17:38:21.736 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-04T17:38:21.752 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T17:38:21.784 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libcommons-parent-java (43-1) ... 2024-03-04T17:38:21.786 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T17:38:21.893 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libsisu-inject-java (0.3.4-2) ... 2024-03-04T17:38:21.994 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up s3cmd (2.2.0-1) ... 2024-03-04T17:38:22.047 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package jq. 2024-03-04T17:38:22.063 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T17:38:22.114 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-04T17:38:22.288 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libsisu-plexus-java (0.3.4-3) ... 2024-03-04T17:38:22.367 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package socat. 2024-03-04T17:38:22.394 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-04T17:38:22.413 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libmaven-resolver-java (1.4.2-3build1) ... 2024-03-04T17:38:22.433 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-04T17:38:22.539 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libguava-java (29.0-6) ... 2024-03-04T17:38:22.665 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libcommons-lang3-java (3.11-1) ... 2024-03-04T17:38:22.737 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package xmlstarlet. 2024-03-04T17:38:22.752 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-04T17:38:22.791 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libjansi-native-java (1.8-1) ... 2024-03-04T17:38:22.795 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-04T17:38:22.917 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libwagon-file-java (3.3.4-1) ... 2024-03-04T17:38:23.043 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libcommons-io-java (2.11.0-2) ... 2024-03-04T17:38:23.144 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libguice-java (4.2.3-2) ... 2024-03-04T17:38:23.183 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-test. 2024-03-04T17:38:23.210 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../097-ceph-test_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:23.245 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libjansi-java (1.18-1) ... 2024-03-04T17:38:23.249 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-test (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:23.355 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libmaven-shared-utils-java (3.3.0-1) ... 2024-03-04T17:38:23.480 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libmaven3-core-java (3.6.3-5) ... 2024-03-04T17:38:23.606 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up maven (3.6.3-5) ... 2024-03-04T17:38:23.872 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/share/maven/bin/mvn to provide /usr/bin/mvn (mvn) in auto mode 2024-03-04T17:38:23.936 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T17:38:25.249 INFO:teuthology.orchestra.run.smithi137.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-04T17:38:25.254 DEBUG:teuthology.parallel:result is None 2024-03-04T17:38:25.628 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-volume. 2024-03-04T17:38:25.655 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-1871-g3aec3903-1jammy_all.deb ... 2024-03-04T17:38:25.694 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-volume (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:25.998 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-04T17:38:26.014 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:26.057 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libcephfs-dev (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:26.368 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-04T17:38:26.396 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-04T17:38:26.434 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T17:38:26.705 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-04T17:38:26.733 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-03-04T17:38:26.763 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-04T17:38:27.118 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package nvme-cli. 2024-03-04T17:38:27.145 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-04T17:38:27.184 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T17:38:27.580 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package pkg-config. 2024-03-04T17:38:27.597 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-04T17:38:27.630 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T17:38:27.875 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-04T17:38:27.891 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-04T17:38:27.933 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-04T17:38:28.336 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-04T17:38:28.364 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-04T17:38:28.402 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-04T17:38:28.731 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pastescript. 2024-03-04T17:38:28.758 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-03-04T17:38:28.796 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-04T17:38:29.117 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pluggy. 2024-03-04T17:38:29.144 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-04T17:38:29.183 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-04T17:38:29.461 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-psutil. 2024-03-04T17:38:29.477 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-04T17:38:29.511 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-04T17:38:29.864 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-py. 2024-03-04T17:38:29.891 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-03-04T17:38:29.930 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-04T17:38:30.251 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pygments. 2024-03-04T17:38:30.278 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-04T17:38:30.317 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T17:38:30.754 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-04T17:38:30.782 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-04T17:38:30.820 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-04T17:38:31.098 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-toml. 2024-03-04T17:38:31.126 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-03-04T17:38:31.165 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-04T17:38:31.485 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pytest. 2024-03-04T17:38:31.512 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-04T17:38:31.551 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T17:38:31.904 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-simplejson. 2024-03-04T17:38:31.932 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-04T17:38:31.971 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-04T17:38:32.248 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-04T17:38:32.276 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-04T17:38:32.323 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-04T17:38:33.035 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package radosgw. 2024-03-04T17:38:33.051 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../116-radosgw_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:33.085 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking radosgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:33.896 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package rbd-fuse. 2024-03-04T17:38:33.913 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-1871-g3aec3903-1jammy_amd64.deb ... 2024-03-04T17:38:33.954 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking rbd-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:34.267 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-04T17:38:34.501 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T17:38:34.602 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T17:38:34.855 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-04T17:38:35.196 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-04T17:38:35.634 INFO:teuthology.orchestra.run.smithi173.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-04T17:38:35.704 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:36.212 INFO:teuthology.orchestra.run.smithi173.stdout:Adding system user cephadm....done 2024-03-04T17:38:36.327 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T17:38:36.594 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-04T17:38:36.845 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-04T17:38:36.971 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-04T17:38:37.239 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-04T17:38:37.499 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T17:38:37.625 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-04T17:38:37.910 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T17:38:38.254 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-04T17:38:38.505 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-04T17:38:38.772 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-04T17:38:39.057 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-04T17:38:39.308 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-04T17:38:39.560 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-04T17:38:39.802 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-ceph-argparse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:40.045 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T17:38:40.171 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T17:38:40.297 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T17:38:40.423 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-04T17:38:40.674 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-04T17:38:41.057 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-04T17:38:41.089 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-04T17:38:41.144 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T17:38:41.261 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-04T17:38:41.613 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T17:38:41.739 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-04T17:38:42.007 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T17:38:42.258 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-04T17:38:42.534 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up zip (3.0-12build2) ... 2024-03-04T17:38:42.635 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T17:38:43.253 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T17:38:43.504 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T17:38:43.630 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-04T17:38:43.756 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T17:38:44.049 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-04T17:38:44.409 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-04T17:38:44.643 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T17:38:45.003 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T17:38:45.296 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T17:38:45.681 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-04T17:38:45.940 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-04T17:38:46.050 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:46.318 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-idna (3.3-1) ... 2024-03-04T17:38:46.619 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T17:38:47.858 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T17:38:48.092 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:38:48.227 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-04T17:38:48.486 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T17:38:48.612 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-04T17:38:48.738 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T17:38:49.031 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-04T17:38:49.291 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-04T17:38:49.542 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:38:49.659 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-04T17:38:49.953 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-04T17:38:50.203 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-04T17:38:50.463 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-04T17:38:50.739 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-04T17:38:51.007 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-04T17:38:51.266 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T17:38:51.367 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-04T17:38:51.627 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T17:38:51.753 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T17:38:52.062 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-04T17:38:52.363 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-04T17:38:52.614 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T17:38:52.734 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-04T17:38:52.860 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T17:38:52.949 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-04T17:38:53.187 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T17:38:53.313 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T17:38:53.723 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-04T17:38:54.000 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-04T17:38:54.109 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-04T17:38:54.344 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T17:38:54.486 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T17:38:54.813 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T17:38:55.114 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T17:38:55.390 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-04T17:38:55.499 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-04T17:38:55.759 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-04T17:38:56.044 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T17:38:56.389 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-04T17:38:56.766 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-04T17:38:57.034 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T17:38:57.345 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T17:38:57.471 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librados2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:57.597 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:57.724 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-04T17:38:58.035 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libcephfs2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:58.161 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libradosstriper1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:58.312 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-04T17:38:58.589 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librbd1 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:58.715 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-modules-core (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:58.833 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:59.036 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T17:38:59.036 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T17:38:59.543 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libcephfs-dev (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:59.652 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rados (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:59.778 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librgw2 (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:38:59.904 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T17:39:01.100 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rbd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:01.210 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up rbd-fuse (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:01.327 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:01.453 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cephfs (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:01.580 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-common (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:02.115 INFO:teuthology.orchestra.run.smithi173.stdout:Adding group ceph....done 2024-03-04T17:39:02.508 INFO:teuthology.orchestra.run.smithi173.stdout:Adding system user ceph....done 2024-03-04T17:39:02.669 INFO:teuthology.orchestra.run.smithi173.stdout:Setting system user ceph properties....done 2024-03-04T17:39:02.688 INFO:teuthology.orchestra.run.smithi173.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-04T17:39:02.826 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-04T17:39:03.161 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-04T17:39:03.665 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-test (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:03.791 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up radosgw (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:04.268 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T17:39:04.268 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T17:39:04.770 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-base (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:05.106 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-04T17:39:05.598 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mds (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:05.931 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T17:39:05.932 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T17:39:06.410 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:06.620 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T17:39:06.620 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T17:39:07.111 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-osd (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:07.399 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T17:39:07.399 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T17:39:07.879 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:08.006 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:08.157 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mon (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:08.362 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T17:39:08.362 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T17:39:08.875 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-cephadm (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:09.009 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:09.135 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-dashboard (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:09.286 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-volume (19.0.0-1871-g3aec3903-1jammy) ... 2024-03-04T17:39:09.640 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-04T17:39:09.765 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-04T17:39:09.910 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T17:39:10.766 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-04T17:39:11.423 INFO:teuthology.orchestra.run.smithi173.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-04T17:39:11.426 DEBUG:teuthology.orchestra.run.smithi173:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install s3cmd maven 2024-03-04T17:39:11.478 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-03-04T17:39:11.665 INFO:teuthology.orchestra.run.smithi173.stdout:Building dependency tree... 2024-03-04T17:39:11.666 INFO:teuthology.orchestra.run.smithi173.stdout:Reading state information... 2024-03-04T17:39:11.820 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T17:39:11.820 INFO:teuthology.orchestra.run.smithi173.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T17:39:11.820 INFO:teuthology.orchestra.run.smithi173.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T17:39:11.820 INFO:teuthology.orchestra.run.smithi173.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T17:39:11.820 INFO:teuthology.orchestra.run.smithi173.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T17:39:11.820 INFO:teuthology.orchestra.run.smithi173.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T17:39:11.821 INFO:teuthology.orchestra.run.smithi173.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T17:39:11.822 INFO:teuthology.orchestra.run.smithi173.stdout:The following additional packages will be installed: 2024-03-04T17:39:11.822 INFO:teuthology.orchestra.run.smithi173.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-03-04T17:39:11.822 INFO:teuthology.orchestra.run.smithi173.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-03-04T17:39:11.822 INFO:teuthology.orchestra.run.smithi173.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-03-04T17:39:11.822 INFO:teuthology.orchestra.run.smithi173.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-03-04T17:39:11.822 INFO:teuthology.orchestra.run.smithi173.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-03-04T17:39:11.822 INFO:teuthology.orchestra.run.smithi173.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-03-04T17:39:11.822 INFO:teuthology.orchestra.run.smithi173.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-03-04T17:39:11.822 INFO:teuthology.orchestra.run.smithi173.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-03-04T17:39:11.822 INFO:teuthology.orchestra.run.smithi173.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-03-04T17:39:11.822 INFO:teuthology.orchestra.run.smithi173.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-03-04T17:39:11.822 INFO:teuthology.orchestra.run.smithi173.stdout: libwagon-http-shaded-java libwagon-provider-api-java python3-magic 2024-03-04T17:39:11.823 INFO:teuthology.orchestra.run.smithi173.stdout:Suggested packages: 2024-03-04T17:39:11.823 INFO:teuthology.orchestra.run.smithi173.stdout: libaopalliance-java-doc libatinject-jsr330-api-java-doc libel-api-java 2024-03-04T17:39:11.823 INFO:teuthology.orchestra.run.smithi173.stdout: libcommons-io-java-doc libcommons-lang3-java-doc libasm-java libcglib-java 2024-03-04T17:39:11.823 INFO:teuthology.orchestra.run.smithi173.stdout: libjsr305-java-doc libmaven-shared-utils-java-doc liblogback-java 2024-03-04T17:39:11.823 INFO:teuthology.orchestra.run.smithi173.stdout: libplexus-classworlds-java-doc libplexus-sec-dispatcher-java-doc 2024-03-04T17:39:11.823 INFO:teuthology.orchestra.run.smithi173.stdout: libplexus-utils2-java-doc testng libcommons-logging-java liblog4j1.2-java 2024-03-04T17:39:11.859 INFO:teuthology.orchestra.run.smithi173.stdout:The following NEW packages will be installed: 2024-03-04T17:39:11.859 INFO:teuthology.orchestra.run.smithi173.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-03-04T17:39:11.859 INFO:teuthology.orchestra.run.smithi173.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-03-04T17:39:11.859 INFO:teuthology.orchestra.run.smithi173.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-03-04T17:39:11.859 INFO:teuthology.orchestra.run.smithi173.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-03-04T17:39:11.860 INFO:teuthology.orchestra.run.smithi173.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-03-04T17:39:11.860 INFO:teuthology.orchestra.run.smithi173.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-03-04T17:39:11.860 INFO:teuthology.orchestra.run.smithi173.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-03-04T17:39:11.860 INFO:teuthology.orchestra.run.smithi173.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-03-04T17:39:11.860 INFO:teuthology.orchestra.run.smithi173.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-03-04T17:39:11.860 INFO:teuthology.orchestra.run.smithi173.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-03-04T17:39:11.860 INFO:teuthology.orchestra.run.smithi173.stdout: libwagon-http-shaded-java libwagon-provider-api-java maven python3-magic 2024-03-04T17:39:11.860 INFO:teuthology.orchestra.run.smithi173.stdout: s3cmd 2024-03-04T17:39:12.047 INFO:teuthology.orchestra.run.smithi173.stdout:0 upgraded, 35 newly installed, 0 to remove and 275 not upgraded. 2024-03-04T17:39:12.047 INFO:teuthology.orchestra.run.smithi173.stdout:Need to get 10.4 MB of archives. 2024-03-04T17:39:12.047 INFO:teuthology.orchestra.run.smithi173.stdout:After this operation, 13.9 MB of additional disk space will be used. 2024-03-04T17:39:12.047 INFO:teuthology.orchestra.run.smithi173.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libapache-pom-java all 18-1 [4,720 B] 2024-03-04T17:39:12.103 INFO:teuthology.orchestra.run.smithi173.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libatinject-jsr330-api-java all 1.0+ds1-5 [5,348 B] 2024-03-04T17:39:12.103 INFO:teuthology.orchestra.run.smithi173.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-interceptor-3.0-spec-java all 1.0.1-4fakesync [8,616 B] 2024-03-04T17:39:12.114 INFO:teuthology.orchestra.run.smithi173.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcdi-api-java all 1.2-3 [54.3 kB] 2024-03-04T17:39:12.215 INFO:teuthology.orchestra.run.smithi173.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-cli-java all 1.4-2 [55.8 kB] 2024-03-04T17:39:12.262 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-parent-java all 43-1 [10.8 kB] 2024-03-04T17:39:12.266 INFO:teuthology.orchestra.run.smithi173.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-io-java all 2.11.0-2 [297 kB] 2024-03-04T17:39:12.356 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-lang3-java all 3.11-1 [526 kB] 2024-03-04T17:39:12.417 INFO:teuthology.orchestra.run.smithi173.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-annotation-1.3-spec-java all 1.3-1 [11.2 kB] 2024-03-04T17:39:12.418 INFO:teuthology.orchestra.run.smithi173.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjsr305-java all 0.1~+svn49-11 [27.0 kB] 2024-03-04T17:39:12.419 INFO:teuthology.orchestra.run.smithi173.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguava-java all 29.0-6 [2,418 kB] 2024-03-04T17:39:12.584 INFO:teuthology.orchestra.run.smithi173.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libaopalliance-java all 20070526-6 [9,084 B] 2024-03-04T17:39:12.584 INFO:teuthology.orchestra.run.smithi173.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguice-java all 4.2.3-2 [1,434 kB] 2024-03-04T17:39:12.612 INFO:teuthology.orchestra.run.smithi173.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libhawtjni-runtime-java all 1.17-1 [28.8 kB] 2024-03-04T17:39:12.613 INFO:teuthology.orchestra.run.smithi173.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-native-java all 1.8-1 [23.8 kB] 2024-03-04T17:39:12.614 INFO:teuthology.orchestra.run.smithi173.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-java all 1.18-1 [56.8 kB] 2024-03-04T17:39:12.615 INFO:teuthology.orchestra.run.smithi173.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-parent-java all 31-2 [5,140 B] 2024-03-04T17:39:12.616 INFO:teuthology.orchestra.run.smithi173.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-utils2-java all 3.3.0-1 [250 kB] 2024-03-04T17:39:12.620 INFO:teuthology.orchestra.run.smithi173.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-provider-api-java all 3.3.4-1 [48.5 kB] 2024-03-04T17:39:12.621 INFO:teuthology.orchestra.run.smithi173.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-resolver-java all 1.4.2-3build1 [555 kB] 2024-03-04T17:39:12.628 INFO:teuthology.orchestra.run.smithi173.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-shared-utils-java all 3.3.0-1 [149 kB] 2024-03-04T17:39:12.644 INFO:teuthology.orchestra.run.smithi173.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-cipher-java all 1.8-2 [15.1 kB] 2024-03-04T17:39:12.644 INFO:teuthology.orchestra.run.smithi173.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-classworlds-java all 2.6.0-1 [49.4 kB] 2024-03-04T17:39:12.645 INFO:teuthology.orchestra.run.smithi173.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-component-annotations-java all 2.1.0-1 [6,564 B] 2024-03-04T17:39:12.646 INFO:teuthology.orchestra.run.smithi173.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-interpolation-java all 1.26-1 [76.8 kB] 2024-03-04T17:39:12.671 INFO:teuthology.orchestra.run.smithi173.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-sec-dispatcher-java all 1.4-4 [28.1 kB] 2024-03-04T17:39:12.672 INFO:teuthology.orchestra.run.smithi173.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libslf4j-java all 1.7.32-1 [141 kB] 2024-03-04T17:39:12.674 INFO:teuthology.orchestra.run.smithi173.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-inject-java all 0.3.4-2 [347 kB] 2024-03-04T17:39:12.680 INFO:teuthology.orchestra.run.smithi173.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-plexus-java all 0.3.4-3 [181 kB] 2024-03-04T17:39:12.683 INFO:teuthology.orchestra.run.smithi173.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven3-core-java all 3.6.3-5 [1,535 kB] 2024-03-04T17:39:12.710 INFO:teuthology.orchestra.run.smithi173.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-file-java all 3.3.4-1 [7,928 B] 2024-03-04T17:39:12.713 INFO:teuthology.orchestra.run.smithi173.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-http-shaded-java all 3.3.4-1 [1,855 kB] 2024-03-04T17:39:12.741 INFO:teuthology.orchestra.run.smithi173.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 maven all 3.6.3-5 [17.8 kB] 2024-03-04T17:39:12.742 INFO:teuthology.orchestra.run.smithi173.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-magic all 2:0.4.24-2 [12.6 kB] 2024-03-04T17:39:12.742 INFO:teuthology.orchestra.run.smithi173.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/universe amd64 s3cmd all 2.2.0-1 [120 kB] 2024-03-04T17:39:13.176 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 10.4 MB in 1s (11.8 MB/s) 2024-03-04T17:39:13.267 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libapache-pom-java. 2024-03-04T17:39:13.327 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173563 files and directories currently installed.) 2024-03-04T17:39:13.332 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../00-libapache-pom-java_18-1_all.deb ... 2024-03-04T17:39:13.383 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libapache-pom-java (18-1) ... 2024-03-04T17:39:13.644 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libatinject-jsr330-api-java. 2024-03-04T17:39:13.660 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../01-libatinject-jsr330-api-java_1.0+ds1-5_all.deb ... 2024-03-04T17:39:13.694 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-03-04T17:39:13.922 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libgeronimo-interceptor-3.0-spec-java. 2024-03-04T17:39:13.938 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../02-libgeronimo-interceptor-3.0-spec-java_1.0.1-4fakesync_all.deb ... 2024-03-04T17:39:13.981 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-03-04T17:39:14.233 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libcdi-api-java. 2024-03-04T17:39:14.249 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../03-libcdi-api-java_1.2-3_all.deb ... 2024-03-04T17:39:14.291 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libcdi-api-java (1.2-3) ... 2024-03-04T17:39:14.553 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libcommons-cli-java. 2024-03-04T17:39:14.581 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../04-libcommons-cli-java_1.4-2_all.deb ... 2024-03-04T17:39:14.619 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libcommons-cli-java (1.4-2) ... 2024-03-04T17:39:14.872 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libcommons-parent-java. 2024-03-04T17:39:14.888 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../05-libcommons-parent-java_43-1_all.deb ... 2024-03-04T17:39:14.930 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libcommons-parent-java (43-1) ... 2024-03-04T17:39:15.183 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libcommons-io-java. 2024-03-04T17:39:15.211 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../06-libcommons-io-java_2.11.0-2_all.deb ... 2024-03-04T17:39:15.249 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libcommons-io-java (2.11.0-2) ... 2024-03-04T17:39:15.511 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libcommons-lang3-java. 2024-03-04T17:39:15.527 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../07-libcommons-lang3-java_3.11-1_all.deb ... 2024-03-04T17:39:15.569 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libcommons-lang3-java (3.11-1) ... 2024-03-04T17:39:15.954 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libgeronimo-annotation-1.3-spec-java. 2024-03-04T17:39:15.982 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../08-libgeronimo-annotation-1.3-spec-java_1.3-1_all.deb ... 2024-03-04T17:39:16.023 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-03-04T17:39:16.285 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libjsr305-java. 2024-03-04T17:39:16.301 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../09-libjsr305-java_0.1~+svn49-11_all.deb ... 2024-03-04T17:39:16.343 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libjsr305-java (0.1~+svn49-11) ... 2024-03-04T17:39:16.579 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libguava-java. 2024-03-04T17:39:16.595 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../10-libguava-java_29.0-6_all.deb ... 2024-03-04T17:39:16.629 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libguava-java (29.0-6) ... 2024-03-04T17:39:17.057 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libaopalliance-java. 2024-03-04T17:39:17.073 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../11-libaopalliance-java_20070526-6_all.deb ... 2024-03-04T17:39:17.115 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libaopalliance-java (20070526-6) ... 2024-03-04T17:39:17.377 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libguice-java. 2024-03-04T17:39:17.393 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../12-libguice-java_4.2.3-2_all.deb ... 2024-03-04T17:39:17.452 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libguice-java (4.2.3-2) ... 2024-03-04T17:39:17.822 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libhawtjni-runtime-java. 2024-03-04T17:39:17.838 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../13-libhawtjni-runtime-java_1.17-1_all.deb ... 2024-03-04T17:39:17.872 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libhawtjni-runtime-java (1.17-1) ... 2024-03-04T17:39:18.108 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libjansi-native-java. 2024-03-04T17:39:18.124 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../14-libjansi-native-java_1.8-1_all.deb ... 2024-03-04T17:39:18.167 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libjansi-native-java (1.8-1) ... 2024-03-04T17:39:18.436 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libjansi-java. 2024-03-04T17:39:18.452 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../15-libjansi-java_1.18-1_all.deb ... 2024-03-04T17:39:18.495 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libjansi-java (1.18-1) ... 2024-03-04T17:39:18.722 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libmaven-parent-java. 2024-03-04T17:39:18.738 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../16-libmaven-parent-java_31-2_all.deb ... 2024-03-04T17:39:18.772 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libmaven-parent-java (31-2) ... 2024-03-04T17:39:18.983 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libplexus-utils2-java. 2024-03-04T17:39:19.011 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../17-libplexus-utils2-java_3.3.0-1_all.deb ... 2024-03-04T17:39:19.041 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libplexus-utils2-java (3.3.0-1) ... 2024-03-04T17:39:19.336 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libwagon-provider-api-java. 2024-03-04T17:39:19.363 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../18-libwagon-provider-api-java_3.3.4-1_all.deb ... 2024-03-04T17:39:19.402 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libwagon-provider-api-java (3.3.4-1) ... 2024-03-04T17:39:19.639 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libmaven-resolver-java. 2024-03-04T17:39:19.655 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../19-libmaven-resolver-java_1.4.2-3build1_all.deb ... 2024-03-04T17:39:19.697 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libmaven-resolver-java (1.4.2-3build1) ... 2024-03-04T17:39:20.001 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libmaven-shared-utils-java. 2024-03-04T17:39:20.017 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../20-libmaven-shared-utils-java_3.3.0-1_all.deb ... 2024-03-04T17:39:20.060 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libmaven-shared-utils-java (3.3.0-1) ... 2024-03-04T17:39:20.329 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libplexus-cipher-java. 2024-03-04T17:39:20.357 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../21-libplexus-cipher-java_1.8-2_all.deb ... 2024-03-04T17:39:20.396 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libplexus-cipher-java (1.8-2) ... 2024-03-04T17:39:20.649 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libplexus-classworlds-java. 2024-03-04T17:39:20.676 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../22-libplexus-classworlds-java_2.6.0-1_all.deb ... 2024-03-04T17:39:20.715 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libplexus-classworlds-java (2.6.0-1) ... 2024-03-04T17:39:20.943 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libplexus-component-annotations-java. 2024-03-04T17:39:20.959 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../23-libplexus-component-annotations-java_2.1.0-1_all.deb ... 2024-03-04T17:39:21.104 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libplexus-component-annotations-java (2.1.0-1) ... 2024-03-04T17:39:21.380 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libplexus-interpolation-java. 2024-03-04T17:39:21.397 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../24-libplexus-interpolation-java_1.26-1_all.deb ... 2024-03-04T17:39:21.439 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libplexus-interpolation-java (1.26-1) ... 2024-03-04T17:39:21.675 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libplexus-sec-dispatcher-java. 2024-03-04T17:39:21.691 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../25-libplexus-sec-dispatcher-java_1.4-4_all.deb ... 2024-03-04T17:39:21.724 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libplexus-sec-dispatcher-java (1.4-4) ... 2024-03-04T17:39:21.961 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libslf4j-java. 2024-03-04T17:39:21.988 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../26-libslf4j-java_1.7.32-1_all.deb ... 2024-03-04T17:39:22.027 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libslf4j-java (1.7.32-1) ... 2024-03-04T17:39:22.307 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libsisu-inject-java. 2024-03-04T17:39:22.334 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../27-libsisu-inject-java_0.3.4-2_all.deb ... 2024-03-04T17:39:22.373 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libsisu-inject-java (0.3.4-2) ... 2024-03-04T17:39:22.651 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libsisu-plexus-java. 2024-03-04T17:39:22.679 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../28-libsisu-plexus-java_0.3.4-3_all.deb ... 2024-03-04T17:39:22.709 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libsisu-plexus-java (0.3.4-3) ... 2024-03-04T17:39:22.971 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libmaven3-core-java. 2024-03-04T17:39:22.998 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../29-libmaven3-core-java_3.6.3-5_all.deb ... 2024-03-04T17:39:23.037 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libmaven3-core-java (3.6.3-5) ... 2024-03-04T17:39:23.425 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libwagon-file-java. 2024-03-04T17:39:23.441 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../30-libwagon-file-java_3.3.4-1_all.deb ... 2024-03-04T17:39:23.483 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libwagon-file-java (3.3.4-1) ... 2024-03-04T17:39:23.745 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libwagon-http-shaded-java. 2024-03-04T17:39:23.772 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../31-libwagon-http-shaded-java_3.3.4-1_all.deb ... 2024-03-04T17:39:23.811 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libwagon-http-shaded-java (3.3.4-1) ... 2024-03-04T17:39:24.206 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package maven. 2024-03-04T17:39:24.234 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../32-maven_3.6.3-5_all.deb ... 2024-03-04T17:39:24.272 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking maven (3.6.3-5) ... 2024-03-04T17:39:24.622 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-magic. 2024-03-04T17:39:24.650 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../33-python3-magic_2%3a0.4.24-2_all.deb ... 2024-03-04T17:39:24.776 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-magic (2:0.4.24-2) ... 2024-03-04T17:39:25.046 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package s3cmd. 2024-03-04T17:39:25.073 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../34-s3cmd_2.2.0-1_all.deb ... 2024-03-04T17:39:25.120 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking s3cmd (2.2.0-1) ... 2024-03-04T17:39:25.492 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libslf4j-java (1.7.32-1) ... 2024-03-04T17:39:25.618 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libplexus-utils2-java (3.3.0-1) ... 2024-03-04T17:39:25.728 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libplexus-classworlds-java (2.6.0-1) ... 2024-03-04T17:39:25.854 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libjsr305-java (0.1~+svn49-11) ... 2024-03-04T17:39:25.980 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libaopalliance-java (20070526-6) ... 2024-03-04T17:39:26.148 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libcommons-cli-java (1.4-2) ... 2024-03-04T17:39:26.291 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libplexus-component-annotations-java (2.1.0-1) ... 2024-03-04T17:39:26.401 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libplexus-cipher-java (1.8-2) ... 2024-03-04T17:39:26.519 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-03-04T17:39:26.628 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-03-04T17:39:26.754 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libapache-pom-java (18-1) ... 2024-03-04T17:39:26.880 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-03-04T17:39:27.006 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libplexus-interpolation-java (1.26-1) ... 2024-03-04T17:39:27.132 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libplexus-sec-dispatcher-java (1.4-4) ... 2024-03-04T17:39:27.258 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libwagon-http-shaded-java (3.3.4-1) ... 2024-03-04T17:39:27.367 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-magic (2:0.4.24-2) ... 2024-03-04T17:39:27.627 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libcdi-api-java (1.2-3) ... 2024-03-04T17:39:27.754 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libhawtjni-runtime-java (1.17-1) ... 2024-03-04T17:39:27.880 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libwagon-provider-api-java (3.3.4-1) ... 2024-03-04T17:39:28.006 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libmaven-parent-java (31-2) ... 2024-03-04T17:39:28.132 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libcommons-parent-java (43-1) ... 2024-03-04T17:39:28.258 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libsisu-inject-java (0.3.4-2) ... 2024-03-04T17:39:28.384 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up s3cmd (2.2.0-1) ... 2024-03-04T17:39:28.669 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libsisu-plexus-java (0.3.4-3) ... 2024-03-04T17:39:28.795 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libmaven-resolver-java (1.4.2-3build1) ... 2024-03-04T17:39:28.946 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libguava-java (29.0-6) ... 2024-03-04T17:39:29.072 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libcommons-lang3-java (3.11-1) ... 2024-03-04T17:39:29.198 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libjansi-native-java (1.8-1) ... 2024-03-04T17:39:29.324 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libwagon-file-java (3.3.4-1) ... 2024-03-04T17:39:29.450 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libcommons-io-java (2.11.0-2) ... 2024-03-04T17:39:29.560 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libguice-java (4.2.3-2) ... 2024-03-04T17:39:29.669 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libjansi-java (1.18-1) ... 2024-03-04T17:39:29.779 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libmaven-shared-utils-java (3.3.0-1) ... 2024-03-04T17:39:29.905 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libmaven3-core-java (3.6.3-5) ... 2024-03-04T17:39:30.031 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up maven (3.6.3-5) ... 2024-03-04T17:39:30.289 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/share/maven/bin/mvn to provide /usr/bin/mvn (mvn) in auto mode 2024-03-04T17:39:30.360 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T17:39:32.196 INFO:teuthology.orchestra.run.smithi173.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-04T17:39:32.202 DEBUG:teuthology.parallel:result is None 2024-03-04T17:39:32.202 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:39:32.550 DEBUG:teuthology.orchestra.run.smithi137:> dpkg-query -W -f '${Version}' ceph 2024-03-04T17:39:32.570 INFO:teuthology.orchestra.run.smithi137.stdout:19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:39:32.570 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:39:32.570 INFO:teuthology.task.install:The correct ceph version 19.0.0-1871-g3aec3903-1jammy is installed. 2024-03-04T17:39:32.572 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:39:32.920 DEBUG:teuthology.orchestra.run.smithi173:> dpkg-query -W -f '${Version}' ceph 2024-03-04T17:39:32.939 INFO:teuthology.orchestra.run.smithi173.stdout:19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:39:32.939 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1871-g3aec3903-1jammy 2024-03-04T17:39:32.939 INFO:teuthology.task.install:The correct ceph version 19.0.0-1871-g3aec3903-1jammy is installed. 2024-03-04T17:39:32.941 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-04T17:39:32.941 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:39:32.941 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T17:39:32.956 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:39:32.956 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T17:39:32.995 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-04T17:39:32.996 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:39:32.996 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T17:39:33.010 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T17:39:33.062 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:39:33.062 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T17:39:33.075 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T17:39:33.131 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-04T17:39:33.131 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:39:33.131 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T17:39:33.144 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T17:39:33.198 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:39:33.198 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T17:39:33.210 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T17:39:33.266 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-04T17:39:33.266 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:39:33.266 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T17:39:33.280 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T17:39:33.334 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:39:33.334 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T17:39:33.347 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T17:39:33.402 INFO:teuthology.run_tasks:Running task ceph... 2024-03-04T17:39:33.481 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-04T17:39:33.481 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /var/log/ceph 2024-03-04T17:39:33.483 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 777 /var/log/ceph 2024-03-04T17:39:33.496 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-04T17:39:33.497 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-04T17:39:33.536 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-04T17:39:33.551 INFO:tasks.ceph:Creating extra log directories... 2024-03-04T17:39:33.551 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-04T17:39:33.592 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-04T17:39:33.616 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-04T17:39:33.616 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw enable ops log': True, 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-04T17:39:33.616 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi/7581361', 'branch': 'wip-rgw-d4n', 'description': 'rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}', 'email': None, 'first_in_suite': False, 'job_id': '7581361', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'suriarte-2024-03-04_17:13:34-rgw-wip-rgw-d4n-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-rgw-d4n'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw enable ops log': True, 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME'], 'sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8'}}, 'openssl_keys': {'rgw.client.0': {'ca': 'root', 'client': 'client.0', 'embed-key': True}, 'root': {'client': 'client.0', 'cn': 'teuthology', 'install': ['client.0'], 'key-type': 'rsa:4096'}}, 'rgw': {'client.0': {'ssl certificate': 'rgw.client.0', 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'compression type': 'random', 'datacache': True, 'datacache_path': '/tmp/rgw_datacache', 'ec-data-pool': False, 'frontend': 'beast', 'storage classes': 'LUKEWARM, FROZEN'}, 's3tests': {'force-branch': 'ceph-master', 'storage classes': 'LUKEWARM, FROZEN'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rgw-d4n', 'sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8'}}, 'owner': 'scheduled_suriarte@teuthology', 'priority': 150, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8', 'sleep_before_teardown': 0, 'suite': 'rgw', 'suite_branch': 'wip-rgw-d4n', 'suite_path': '/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/pritha-srivastava/ceph.git', 'suite_sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8', 'targets': {'smithi137.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwf7HDKkJ1igXST3tKMyYd6HtpysF747RRvrRt1UhlAM+YSj5Ofqp9m/IPxkJnFcDUuuMUvWFaaT2TvvPJGyk4=', 'smithi173.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEDaCoKsQFWnbYcwH2ctXsZs8ZySlgfJZBOEhhmGWx4CEbqa+R0+ZUhWG8fplq/qksL24GS9lA3riaBlLKh5/aY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'flavor': 'default', 'sha1': '3aec390336fdfdc8f927818b718cfb0361bd47c8'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': None}}, {'tox': ['client.0']}, {'workunit': {'clients': {'client.0': ['rgw/run-datacache.sh']}, 'env': {'RGW_DATACACHE_PATH': '/tmp/rgw_datacache'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-bucket-check.sh']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh']}}}, {'workunit': {'clients': {'client.0': ['rgw/test_rgw_s3_mp_reupload.sh']}}}, {'ragweed': {'client.0': {'default-branch': 'ceph-master', 'rgw_server': 'client.0', 'stages': 'prepare,check'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-reshard.sh']}}}, {'s3tests-java': {'client.0': {'force-branch': 'ceph-master', 'force-repo': 'https://github.com/ceph/java_s3tests.git'}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-versioning.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b1dac5519c57c269ea98a22fb7729016a1d0e4d2', 'timestamp': '2024-03-04_17:13:34', 'tube': 'smithi', 'user': 'suriarte', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-04T17:39:33.617 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-04T17:39:33.648 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-04T17:39:33.656 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m0777 -- /var/run/ceph 2024-03-04T17:39:33.696 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m0777 -- /var/run/ceph 2024-03-04T17:39:33.711 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:39:33.711 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T17:39:33.755 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T17:39:33.755 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-03-04T17:39:33.804 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T17:39:33.804 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:39:33.804 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 883 Links: 1 2024-03-04T17:39:33.804 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:39:33.804 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-04 17:32:50.352857390 +0000 2024-03-04T17:39:33.804 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-04 17:32:50.100859638 +0000 2024-03-04T17:39:33.804 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-04 17:32:50.100859638 +0000 2024-03-04T17:39:33.804 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-03-04T17:39:33.804 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T17:39:33.858 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-04T17:39:33.858 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-04T17:39:33.858 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.00043963 s, 1.2 MB/s 2024-03-04T17:39:33.860 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T17:39:33.910 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-03-04T17:39:33.956 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T17:39:33.956 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:39:33.956 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 899 Links: 1 2024-03-04T17:39:33.956 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:39:33.957 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-04 17:32:50.860852858 +0000 2024-03-04T17:39:33.957 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-04 17:32:50.592855248 +0000 2024-03-04T17:39:33.957 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-04 17:32:50.592855248 +0000 2024-03-04T17:39:33.957 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-03-04T17:39:33.957 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T17:39:34.009 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-04T17:39:34.009 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-04T17:39:34.009 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000310938 s, 1.6 MB/s 2024-03-04T17:39:34.010 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T17:39:34.060 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-03-04T17:39:34.107 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T17:39:34.108 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:39:34.108 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 916 Links: 1 2024-03-04T17:39:34.108 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:39:34.108 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-04 17:32:51.376848255 +0000 2024-03-04T17:39:34.108 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-04 17:32:51.100850717 +0000 2024-03-04T17:39:34.108 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-04 17:32:51.100850717 +0000 2024-03-04T17:39:34.108 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-03-04T17:39:34.108 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T17:39:34.161 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-04T17:39:34.161 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-04T17:39:34.161 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000358144 s, 1.4 MB/s 2024-03-04T17:39:34.162 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T17:39:34.208 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-03-04T17:39:34.255 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T17:39:34.256 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:39:34.256 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 940 Links: 1 2024-03-04T17:39:34.256 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:39:34.256 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-04 17:32:51.892843652 +0000 2024-03-04T17:39:34.256 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-04 17:32:51.624846042 +0000 2024-03-04T17:39:34.256 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-04 17:32:51.624846042 +0000 2024-03-04T17:39:34.256 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-03-04T17:39:34.256 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T17:39:34.310 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-04T17:39:34.310 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-04T17:39:34.310 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000385244 s, 1.3 MB/s 2024-03-04T17:39:34.311 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T17:39:34.361 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T17:39:34.361 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:39:34.361 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T17:39:34.367 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T17:39:34.367 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 2024-03-04T17:39:34.416 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T17:39:34.416 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:39:34.416 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 883 Links: 1 2024-03-04T17:39:34.416 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:39:34.416 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-03-04 17:32:43.162390103 +0000 2024-03-04T17:39:34.416 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-03-04 17:32:42.910392354 +0000 2024-03-04T17:39:34.416 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-03-04 17:32:42.910392354 +0000 2024-03-04T17:39:34.416 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2024-03-04T17:39:34.417 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T17:39:34.470 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-03-04T17:39:34.470 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-03-04T17:39:34.470 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000330955 s, 1.5 MB/s 2024-03-04T17:39:34.471 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T17:39:34.521 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 2024-03-04T17:39:34.567 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T17:39:34.567 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:39:34.567 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 894 Links: 1 2024-03-04T17:39:34.567 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:39:34.567 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-03-04 17:32:43.630385922 +0000 2024-03-04T17:39:34.568 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-03-04 17:32:43.374388208 +0000 2024-03-04T17:39:34.568 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-03-04 17:32:43.374388208 +0000 2024-03-04T17:39:34.568 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2024-03-04T17:39:34.568 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T17:39:34.621 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-03-04T17:39:34.621 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-03-04T17:39:34.621 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000436123 s, 1.2 MB/s 2024-03-04T17:39:34.623 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T17:39:34.673 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 2024-03-04T17:39:34.719 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T17:39:34.720 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:39:34.720 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 916 Links: 1 2024-03-04T17:39:34.720 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:39:34.720 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-03-04 17:32:43.870383778 +0000 2024-03-04T17:39:34.720 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-03-04 17:32:43.866383814 +0000 2024-03-04T17:39:34.720 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-03-04 17:32:43.866383814 +0000 2024-03-04T17:39:34.720 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2024-03-04T17:39:34.720 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T17:39:34.774 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-03-04T17:39:34.774 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-03-04T17:39:34.774 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000378663 s, 1.4 MB/s 2024-03-04T17:39:34.776 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T17:39:34.825 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 2024-03-04T17:39:34.871 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T17:39:34.871 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T17:39:34.871 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 934 Links: 1 2024-03-04T17:39:34.871 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T17:39:34.871 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-03-04 17:32:44.590377348 +0000 2024-03-04T17:39:34.871 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-03-04 17:32:44.362379384 +0000 2024-03-04T17:39:34.872 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-03-04 17:32:44.362379384 +0000 2024-03-04T17:39:34.872 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2024-03-04T17:39:34.872 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T17:39:34.926 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-03-04T17:39:34.926 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-03-04T17:39:34.926 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.00037239 s, 1.4 MB/s 2024-03-04T17:39:34.927 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T17:39:34.979 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T17:39:34.979 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi137.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi173.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-04T17:39:34.979 INFO:tasks.ceph:Generating config... 2024-03-04T17:39:34.980 INFO:tasks.ceph:[client] debug rgw = 20 2024-03-04T17:39:34.980 INFO:tasks.ceph:[client] rgw bucket counters cache = True 2024-03-04T17:39:34.980 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2024-03-04T17:39:34.980 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2024-03-04T17:39:34.980 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-03-04T17:39:34.980 INFO:tasks.ceph:[client] rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2024-03-04T17:39:34.980 INFO:tasks.ceph:[client] rgw d3n l1 datacache size = 10737418240 2024-03-04T17:39:34.981 INFO:tasks.ceph:[client] rgw d3n l1 local datacache enabled = True 2024-03-04T17:39:34.981 INFO:tasks.ceph:[client] rgw enable ops log = True 2024-03-04T17:39:34.981 INFO:tasks.ceph:[client] rgw torrent flag = True 2024-03-04T17:39:34.981 INFO:tasks.ceph:[client] rgw user counters cache = True 2024-03-04T17:39:34.981 INFO:tasks.ceph:[client] setgroup = ceph 2024-03-04T17:39:34.981 INFO:tasks.ceph:[client] setuser = ceph 2024-03-04T17:39:34.981 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-03-04T17:39:34.981 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-03-04T17:39:34.981 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2024-03-04T17:39:34.981 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2024-03-04T17:39:34.981 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2024-03-04T17:39:34.981 INFO:tasks.ceph:[global] rgw override bucket index max shards = 1 2024-03-04T17:39:34.981 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-04T17:39:34.981 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-04T17:39:34.981 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-04T17:39:34.981 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-04T17:39:34.981 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-04T17:39:34.982 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] osd fast shutdown = False 2024-03-04T17:39:34.982 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-04T17:39:34.983 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-03-04T17:39:34.983 INFO:tasks.ceph:Setting up mon.a... 2024-03-04T17:39:34.983 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-04T17:39:35.017 INFO:teuthology.orchestra.run.smithi137.stdout:creating /etc/ceph/ceph.keyring 2024-03-04T17:39:35.038 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-04T17:39:35.112 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-04T17:39:35.166 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.137'), ('mon.c', '[v2:172.21.15.137:3301,v1:172.21.15.137:6790]'), ('mon.b', '172.21.15.173')] 2024-03-04T17:39:35.166 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.137,[v2:172.21.15.137:3301,v1:172.21.15.137:6790],172.21.15.173', 'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon osd crush smoke test': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': True, 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-03-04T17:39:35.167 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:39:35.167 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-04T17:39:35.211 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.137 --addv c '[v2:172.21.15.137:3301,v1:172.21.15.137:6790]' --add b 172.21.15.173 --print /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:39:35.297 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:39:35.297 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:39:35.297 INFO:teuthology.orchestra.run.smithi137.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:39:35.297 INFO:teuthology.orchestra.run.smithi137.stdout:monmaptool: generated fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T17:39:35.297 INFO:teuthology.orchestra.run.smithi137.stdout:setting min_mon_release = quincy 2024-03-04T17:39:35.298 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 0 2024-03-04T17:39:35.298 INFO:teuthology.orchestra.run.smithi137.stdout:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T17:39:35.298 INFO:teuthology.orchestra.run.smithi137.stdout:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T17:39:35.298 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-03-04T17:39:35.298234+0000 2024-03-04T17:39:35.298 INFO:teuthology.orchestra.run.smithi137.stdout:min_mon_release 17 (quincy) 2024-03-04T17:39:35.298 INFO:teuthology.orchestra.run.smithi137.stdout:election_strategy: 1 2024-03-04T17:39:35.298 INFO:teuthology.orchestra.run.smithi137.stdout:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T17:39:35.298 INFO:teuthology.orchestra.run.smithi137.stdout:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T17:39:35.298 INFO:teuthology.orchestra.run.smithi137.stdout:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T17:39:35.298 INFO:teuthology.orchestra.run.smithi137.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-04T17:39:35.299 DEBUG:teuthology.orchestra.run.smithi137:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-04T17:39:35.348 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 4e7d04e9-67dd-413e-a1a3-7566f727539e... 2024-03-04T17:39:35.349 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-04T17:39:35.392 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout:[global] 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout: chdir = "" 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout: auth supported = cephx 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout: filestore xattr use omap = true 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout: mon clock drift allowed = 1.000 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout: osd crush chooseleaf type = 0 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout: auth debug = true 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout: ms die on old message = true 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout: ms die on bug = true 2024-03-04T17:39:35.421 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: mon pg warn max object skew = 0 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: # disable pg_autoscaler by default for new pools 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: osd pool default size = 2 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd allow primary affinity = true 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd allow pg remap = true 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on legacy crush tunables = false 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on crush straw calc version zero = false 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on no sortbitwise = false 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on osd down out interval zero = false 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on too few osds = false 2024-03-04T17:39:35.422 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: mon_allow_pool_size_one = true 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: osd default data pool replay window = 5 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: mon allow pool delete = true 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: mon cluster log file level = debug 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: debug asserts on shutdown = true 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: mon health detail to clog = false 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: mon host = "172.21.15.137,[v2:172.21.15.137:3301,v1:172.21.15.137:6790],172.21.15.173" 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: mon client directed command retry = 5 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: ms inject socket failures = 5000 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: osd heartbeat grace = 40 2024-03-04T17:39:35.423 INFO:teuthology.orchestra.run.smithi137.stdout: osd_max_pg_log_entries = 10 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout: osd_min_pg_log_entries = 10 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout: rgw override bucket index max shards = 1 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout: fsid = 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout:[osd] 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout: osd journal size = 100 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout: osd scrub load threshold = 5.0 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout: osd scrub max interval = 600 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout: osd mclock profile = high_recovery_ops 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout: osd recover clone overlap = true 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout: osd recovery max chunk = 1048576 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.424 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug shutdown = true 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug op order = true 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug verify stray on activate = true 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug trim objects = true 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: osd open classes on start = true 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug pg log writeout = true 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: osd deep scrub update digest min age = 30 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: osd map max advance = 10 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: journal zero on create = true 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: filestore ondisk finisher threads = 3 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: filestore apply finisher threads = 3 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.425 INFO:teuthology.orchestra.run.smithi137.stdout: bdev debug aio = true 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug misdirected ops = true 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: bdev async discard = True 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: bdev enable discard = True 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore allocator = bitmap 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore block size = 96636764160 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore fsck on mount = True 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: debug bluefs = 1/20 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: debug bluestore = 1/20 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: debug osd = 20 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: debug rocksdb = 4/10 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd backfillfull_ratio = 0.85 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd full ratio = 0.9 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd nearfull ratio = 0.8 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: osd failsafe full ratio = 0.95 2024-03-04T17:39:35.426 INFO:teuthology.orchestra.run.smithi137.stdout: osd fast shutdown = False 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: osd objectstore = bluestore 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: osd shutdown pgref assert = True 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr] 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: debug mgr = 20 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: debug mon = 20 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: debug auth = 20 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min pgs per osd = 4 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min bytes per osd = 10 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: mgr/telemetry/nag = false 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout:[mon] 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: debug mon = 20 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: debug paxos = 20 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: debug auth = 20 2024-03-04T17:39:35.427 INFO:teuthology.orchestra.run.smithi137.stdout: mon data avail warn = 5 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: mon mgr mkfs grace = 240 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min pgs per osd = 4 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd reporter subtree level = osd 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd prime pg temp = true 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min bytes per osd = 10 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: auth mon ticket ttl = 660 # 11m 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: auth service ticket ttl = 240 # 4m 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: # don't complain about insecure global_id in the test suite 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: # 1m isn't quite enough 2024-03-04T17:39:35.428 INFO:teuthology.orchestra.run.smithi137.stdout: mon_down_mkfs_grace = 2m 2024-03-04T17:39:35.429 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.429 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_filestore_osds = false 2024-03-04T17:39:35.429 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd crush smoke test = False 2024-03-04T17:39:35.429 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:39:35.429 INFO:teuthology.orchestra.run.smithi137.stdout:[client] 2024-03-04T17:39:35.429 INFO:teuthology.orchestra.run.smithi137.stdout: rgw cache enabled = true 2024-03-04T17:39:35.429 INFO:teuthology.orchestra.run.smithi137.stdout: rgw enable ops log = True 2024-03-04T17:39:35.429 INFO:teuthology.orchestra.run.smithi137.stdout: rgw enable usage log = true 2024-03-04T17:39:35.429 INFO:teuthology.orchestra.run.smithi137.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-04T17:39:35.429 INFO:teuthology.orchestra.run.smithi137.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-04T17:39:35.429 INFO:teuthology.orchestra.run.smithi137.stdout: debug rgw = 20 2024-03-04T17:39:35.429 INFO:teuthology.orchestra.run.smithi137.stdout: rgw bucket counters cache = True 2024-03-04T17:39:35.429 INFO:teuthology.orchestra.run.smithi137.stdout: rgw crypt require ssl = False 2024-03-04T17:39:35.429 INFO:teuthology.orchestra.run.smithi137.stdout: rgw crypt s3 kms backend = testing 2024-03-04T17:39:35.430 INFO:teuthology.orchestra.run.smithi137.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-03-04T17:39:35.430 INFO:teuthology.orchestra.run.smithi137.stdout: rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2024-03-04T17:39:35.430 INFO:teuthology.orchestra.run.smithi137.stdout: rgw d3n l1 datacache size = 10737418240 2024-03-04T17:39:35.430 INFO:teuthology.orchestra.run.smithi137.stdout: rgw d3n l1 local datacache enabled = True 2024-03-04T17:39:35.430 INFO:teuthology.orchestra.run.smithi137.stdout: rgw torrent flag = True 2024-03-04T17:39:35.430 INFO:teuthology.orchestra.run.smithi137.stdout: rgw user counters cache = True 2024-03-04T17:39:35.430 INFO:teuthology.orchestra.run.smithi137.stdout: setgroup = ceph 2024-03-04T17:39:35.430 INFO:teuthology.orchestra.run.smithi137.stdout: setuser = ceph 2024-03-04T17:39:35.430 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.a] 2024-03-04T17:39:35.430 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.c] 2024-03-04T17:39:35.430 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.b] 2024-03-04T17:39:35.431 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2024-03-04T17:39:35.431 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2024-03-04T17:39:35.431 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-04T17:39:35.431 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2024-03-04T17:39:35.431 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.431 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2024-03-04T17:39:35.431 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.431 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2024-03-04T17:39:35.431 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.431 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2024-03-04T17:39:35.431 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2024-03-04T17:39:35.432 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.432 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2024-03-04T17:39:35.432 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2024-03-04T17:39:35.432 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.432 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-04T17:39:35.432 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2024-03-04T17:39:35.432 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.432 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2024-03-04T17:39:35.432 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-04T17:39:35.432 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.432 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2024-03-04T17:39:35.432 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.432 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.137,[v2:172.21.15.137:3301,v1:172.21.15.137:6790],172.21.15.173" 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout: mon client directed command retry = 5 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout: ms inject socket failures = 5000 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout: osd heartbeat grace = 40 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout: osd_max_pg_log_entries = 10 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout: osd_min_pg_log_entries = 10 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout: rgw override bucket index max shards = 1 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.434 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd mclock profile = high_recovery_ops 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug trim objects = true 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 1/20 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 1/20 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 20 2024-03-04T17:39:35.436 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 4/10 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: osd fast shutdown = False 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: osd shutdown pgref assert = True 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2024-03-04T17:39:35.437 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: auth mon ticket ttl = 660 # 11m 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: auth service ticket ttl = 240 # 4m 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.438 INFO:teuthology.orchestra.run.smithi173.stdout: # don't complain about insecure global_id in the test suite 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: # 1m isn't quite enough 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: mon_down_mkfs_grace = 2m 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_filestore_osds = false 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd crush smoke test = False 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = True 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-04T17:39:35.439 INFO:teuthology.orchestra.run.smithi173.stdout: debug rgw = 20 2024-03-04T17:39:35.440 INFO:teuthology.orchestra.run.smithi173.stdout: rgw bucket counters cache = True 2024-03-04T17:39:35.440 INFO:teuthology.orchestra.run.smithi173.stdout: rgw crypt require ssl = False 2024-03-04T17:39:35.440 INFO:teuthology.orchestra.run.smithi173.stdout: rgw crypt s3 kms backend = testing 2024-03-04T17:39:35.440 INFO:teuthology.orchestra.run.smithi173.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-03-04T17:39:35.440 INFO:teuthology.orchestra.run.smithi173.stdout: rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2024-03-04T17:39:35.440 INFO:teuthology.orchestra.run.smithi173.stdout: rgw d3n l1 datacache size = 10737418240 2024-03-04T17:39:35.440 INFO:teuthology.orchestra.run.smithi173.stdout: rgw d3n l1 local datacache enabled = True 2024-03-04T17:39:35.440 INFO:teuthology.orchestra.run.smithi173.stdout: rgw torrent flag = True 2024-03-04T17:39:35.440 INFO:teuthology.orchestra.run.smithi173.stdout: rgw user counters cache = True 2024-03-04T17:39:35.440 INFO:teuthology.orchestra.run.smithi173.stdout: setgroup = ceph 2024-03-04T17:39:35.440 INFO:teuthology.orchestra.run.smithi173.stdout: setuser = ceph 2024-03-04T17:39:35.440 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2024-03-04T17:39:35.440 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2024-03-04T17:39:35.440 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2024-03-04T17:39:35.442 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-04T17:39:35.442 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-04T17:39:35.531 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-04T17:39:35.531 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:39:35.532 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-04T17:39:35.537 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:39:35.537 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-04T17:39:35.583 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi137.front.sepia.ceph.com 2024-03-04T17:39:35.583 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:39:35.583 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-04T17:39:35.583 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-04T17:39:35.647 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:39:35.648 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:39:35.695 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi173.front.sepia.ceph.com 2024-03-04T17:39:35.695 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:39:35.695 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-04T17:39:35.695 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-04T17:39:35.719 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:39:35.719 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:39:35.768 INFO:tasks.ceph:Setting up mon nodes... 2024-03-04T17:39:35.768 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-04T17:39:35.768 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-04T17:39:35.811 INFO:teuthology.orchestra.run.smithi137.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-04T17:39:35.814 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-04T17:39:35.857 INFO:teuthology.orchestra.run.smithi173.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-04T17:39:35.868 INFO:tasks.ceph:Setting up mds nodes... 2024-03-04T17:39:35.868 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-04T17:39:35.869 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-04T17:39:35.904 INFO:teuthology.orchestra.run.smithi137.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-04T17:39:35.914 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-04T17:39:35.945 INFO:teuthology.orchestra.run.smithi173.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-04T17:39:35.956 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-04T17:39:35.956 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi137.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi173.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-04T17:39:35.956 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-04T17:39:35.968 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T17:39:35.968 INFO:tasks.ceph:role: osd.0 2024-03-04T17:39:35.968 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi137.front.sepia.ceph.com 2024-03-04T17:39:35.969 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-04T17:39:36.030 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:39:36.030 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:39:36.031 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:39:36.031 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:39:36.031 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:39:36.031 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:39:36.031 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:39:36.031 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:39:36.031 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:39:36.031 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:39:36.498 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-03-04T17:39:36.500 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-03-04T17:39:36.500 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-04T17:39:36.620 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-04T17:39:36.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:39:36.630 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:39:36.631 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-04T17:39:36.687 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T17:39:36.687 INFO:tasks.ceph:role: osd.1 2024-03-04T17:39:36.687 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi137.front.sepia.ceph.com 2024-03-04T17:39:36.687 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-04T17:39:36.750 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:39:36.750 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:39:36.750 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:39:36.750 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:39:36.750 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:39:36.750 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:39:36.750 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:39:36.750 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:39:36.750 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:39:36.750 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:39:37.175 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-03-04T17:39:37.178 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-03-04T17:39:37.178 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-04T17:39:37.242 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-04T17:39:37.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:39:37.292 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:39:37.292 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-04T17:39:37.344 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T17:39:37.344 INFO:tasks.ceph:role: osd.2 2024-03-04T17:39:37.345 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi137.front.sepia.ceph.com 2024-03-04T17:39:37.345 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-04T17:39:37.406 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:39:37.406 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:39:37.406 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:39:37.406 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:39:37.406 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:39:37.406 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:39:37.406 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:39:37.406 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:39:37.406 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:39:37.407 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:39:37.871 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-03-04T17:39:37.873 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-03-04T17:39:37.874 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-04T17:39:37.957 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-04T17:39:38.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:39:38.008 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:39:38.008 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-04T17:39:38.060 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T17:39:38.060 INFO:tasks.ceph:role: osd.3 2024-03-04T17:39:38.061 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi137.front.sepia.ceph.com 2024-03-04T17:39:38.061 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-04T17:39:38.122 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:39:38.123 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:39:38.123 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:39:38.123 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:39:38.123 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:39:38.123 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:39:38.123 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:39:38.123 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:39:38.123 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:39:38.123 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:39:38.595 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-03-04T17:39:38.598 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-03-04T17:39:38.598 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-04T17:39:38.727 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-04T17:39:38.743 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:39:38.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:39:38.744 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:39:38.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:38.857+0000 7fabb6ea57c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-04T17:39:38.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:38.861+0000 7fabb6ea57c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-04T17:39:38.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:38.861+0000 7fabb6ea57c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-04T17:39:38.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:38.861+0000 7fabb6ea57c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-04T17:39:38.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:38.861+0000 7fabb6ea57c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-04T17:39:38.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:38.865+0000 7fabb6ea57c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-04T17:39:41.845 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-04T17:39:41.858 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:39:41.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:41.937+0000 7f2ce64557c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-04T17:39:41.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:41.937+0000 7f2ce64557c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-04T17:39:41.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:41.937+0000 7f2ce64557c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-04T17:39:41.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:41.937+0000 7f2ce64557c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-04T17:39:41.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:41.937+0000 7f2ce64557c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-04T17:39:41.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:41.937+0000 7f2ce64557c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-04T17:39:44.937 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-04T17:39:44.992 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:39:45.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:45.069+0000 7f66c6c967c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-04T17:39:45.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:45.069+0000 7f66c6c967c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-04T17:39:45.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:45.069+0000 7f66c6c967c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-04T17:39:45.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:45.069+0000 7f66c6c967c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-04T17:39:45.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:45.069+0000 7f66c6c967c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-04T17:39:45.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:45.073+0000 7f66c6c967c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-04T17:39:48.051 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-04T17:39:48.111 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:39:48.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:48.189+0000 7f635f10d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-04T17:39:48.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:48.189+0000 7f635f10d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-04T17:39:48.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:48.189+0000 7f635f10d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-04T17:39:48.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:48.189+0000 7f635f10d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-04T17:39:48.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:48.189+0000 7f635f10d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-04T17:39:48.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:39:48.189+0000 7f635f10d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-04T17:39:51.173 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-04T17:39:51.228 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-04T17:39:51.245 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T17:39:51.245 INFO:tasks.ceph:role: osd.4 2024-03-04T17:39:51.245 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi173.front.sepia.ceph.com 2024-03-04T17:39:51.245 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-04T17:39:51.307 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:39:51.307 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:39:51.307 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:39:51.307 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:39:51.307 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:39:51.307 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:39:51.307 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:39:51.307 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:39:51.307 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:39:51.307 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:39:51.796 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2024-03-04T17:39:51.798 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2024-03-04T17:39:51.798 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-04T17:39:51.945 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-04T17:39:51.956 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:39:51.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:39:51.957 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-04T17:39:52.012 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T17:39:52.012 INFO:tasks.ceph:role: osd.5 2024-03-04T17:39:52.012 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi173.front.sepia.ceph.com 2024-03-04T17:39:52.012 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-04T17:39:52.077 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:39:52.077 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:39:52.077 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:39:52.077 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:39:52.077 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:39:52.077 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:39:52.077 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:39:52.077 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:39:52.077 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:39:52.077 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:39:52.526 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2024-03-04T17:39:52.528 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2024-03-04T17:39:52.528 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-04T17:39:52.620 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-04T17:39:52.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:39:52.630 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:39:52.631 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-04T17:39:52.683 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T17:39:52.683 INFO:tasks.ceph:role: osd.6 2024-03-04T17:39:52.683 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi173.front.sepia.ceph.com 2024-03-04T17:39:52.683 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-04T17:39:52.752 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:39:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:39:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:39:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:39:52.752 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:39:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:39:52.752 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:39:52.752 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:39:52.752 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:39:52.752 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:39:53.214 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2024-03-04T17:39:53.216 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2024-03-04T17:39:53.216 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-04T17:39:53.328 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-04T17:39:53.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:39:53.338 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:39:53.338 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-04T17:39:53.392 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T17:39:53.392 INFO:tasks.ceph:role: osd.7 2024-03-04T17:39:53.392 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi173.front.sepia.ceph.com 2024-03-04T17:39:53.393 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-04T17:39:53.455 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T17:39:53.455 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T17:39:53.456 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T17:39:53.456 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T17:39:53.456 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T17:39:53.456 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2024-03-04T17:39:53.456 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T17:39:53.456 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T17:39:53.456 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T17:39:53.456 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T17:39:53.956 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2024-03-04T17:39:53.958 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2024-03-04T17:39:53.958 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-04T17:39:54.088 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-04T17:39:54.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:39:54.099 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T17:39:54.099 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:39:54.201 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:39:54.197+0000 7f20908237c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-04T17:39:54.205 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:39:54.201+0000 7f20908237c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-04T17:39:54.205 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:39:54.201+0000 7f20908237c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-04T17:39:54.205 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:39:54.201+0000 7f20908237c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-04T17:39:54.205 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:39:54.201+0000 7f20908237c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-04T17:39:54.212 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:39:54.209+0000 7f20908237c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-04T17:39:57.196 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-04T17:39:57.210 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:39:57.289 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:39:57.285+0000 7facdc6a87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-04T17:39:57.290 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:39:57.285+0000 7facdc6a87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-04T17:39:57.290 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:39:57.285+0000 7facdc6a87c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-04T17:39:57.290 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:39:57.285+0000 7facdc6a87c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-04T17:39:57.290 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:39:57.285+0000 7facdc6a87c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-04T17:39:57.290 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:39:57.285+0000 7facdc6a87c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-04T17:40:00.286 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-04T17:40:00.339 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:40:00.418 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:40:00.413+0000 7ffa1bed67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-04T17:40:00.418 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:40:00.413+0000 7ffa1bed67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-04T17:40:00.419 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:40:00.413+0000 7ffa1bed67c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-04T17:40:00.419 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:40:00.413+0000 7ffa1bed67c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-04T17:40:00.419 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:40:00.413+0000 7ffa1bed67c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-04T17:40:00.419 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:40:00.413+0000 7ffa1bed67c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-04T17:40:03.402 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-04T17:40:03.457 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:40:03.538 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:40:03.533+0000 7f6f7bfb17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-04T17:40:03.539 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:40:03.533+0000 7f6f7bfb17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-04T17:40:03.539 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:40:03.533+0000 7f6f7bfb17c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-04T17:40:03.539 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:40:03.533+0000 7f6f7bfb17c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-04T17:40:03.539 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:40:03.533+0000 7f6f7bfb17c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-04T17:40:03.539 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-04T17:40:03.533+0000 7f6f7bfb17c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-04T17:40:06.511 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-04T17:40:06.568 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-04T17:40:06.569 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:40:06.569 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-04T17:40:06.583 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:40:06.583 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-04T17:40:06.640 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:40:06.640 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-04T17:40:06.694 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:40:06.695 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-04T17:40:06.746 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:40:06.746 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-04T17:40:06.798 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:40:06.798 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-04T17:40:06.811 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:40:06.812 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-04T17:40:06.867 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:40:06.867 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-04T17:40:06.919 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:40:06.919 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-04T17:40:06.976 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:40:06.976 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-04T17:40:07.032 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:40:07.032 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-04T17:40:07.037 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:40:07.038 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-04T17:40:07.081 INFO:tasks.ceph:Adding keys to all mons... 2024-03-04T17:40:07.081 DEBUG:teuthology.orchestra.run.smithi137:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-04T17:40:07.083 DEBUG:teuthology.orchestra.run.smithi173:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-04T17:40:07.127 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr.y] 2024-03-04T17:40:07.127 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBXB+ZlOulrMBAAh6kSUBlUOETS4nxFRzR1Lw== 2024-03-04T17:40:07.127 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.0] 2024-03-04T17:40:07.127 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBaB+ZlKp9OMxAAsAbeKZ5CTFx9OmtToyMMZw== 2024-03-04T17:40:07.127 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.1] 2024-03-04T17:40:07.127 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBdB+ZlV18rOBAAQalF1widL7G6J7Mztegu/A== 2024-03-04T17:40:07.127 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.2] 2024-03-04T17:40:07.127 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBhB+ZlBCB5BBAAcPdldNaWd2lMrlnbRN/P4A== 2024-03-04T17:40:07.127 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.3] 2024-03-04T17:40:07.127 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBkB+Zlc3eVCxAAPilyLSG+OX/mhnFg8OUliw== 2024-03-04T17:40:07.127 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr.x] 2024-03-04T17:40:07.127 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBXB+Zljt40MxAAMHpruRoL734TTSyfrrJ/Bw== 2024-03-04T17:40:07.127 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.4] 2024-03-04T17:40:07.127 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBqB+ZlscIcDBAAbC8/Rc0BhsSCqpVGfQc6RA== 2024-03-04T17:40:07.128 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.5] 2024-03-04T17:40:07.128 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBtB+ZloTNLERAArlCRCzKZjALYLcVfst92zQ== 2024-03-04T17:40:07.128 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.6] 2024-03-04T17:40:07.128 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBwB+ZlPNH8GBAAURssiBm3D/qerITPpfKr4A== 2024-03-04T17:40:07.128 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.7] 2024-03-04T17:40:07.128 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBzB+ZlbQAkIBAAc2pDk1VEHRmcpV3ZEmH8hQ== 2024-03-04T17:40:07.128 INFO:teuthology.orchestra.run.smithi137.stdout:[client.0] 2024-03-04T17:40:07.128 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBXB+Zle+D3NRAAuvGQwDami6GjsV3Uk2IKgg== 2024-03-04T17:40:07.128 INFO:teuthology.orchestra.run.smithi137.stdout:[client.1] 2024-03-04T17:40:07.128 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBXB+ZlgJd7OBAAjaICGPAjpLEziTvz81jSpQ== 2024-03-04T17:40:07.135 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr.y] 2024-03-04T17:40:07.135 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBXB+ZlOulrMBAAh6kSUBlUOETS4nxFRzR1Lw== 2024-03-04T17:40:07.135 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.0] 2024-03-04T17:40:07.135 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBaB+ZlKp9OMxAAsAbeKZ5CTFx9OmtToyMMZw== 2024-03-04T17:40:07.135 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.1] 2024-03-04T17:40:07.135 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBdB+ZlV18rOBAAQalF1widL7G6J7Mztegu/A== 2024-03-04T17:40:07.135 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.2] 2024-03-04T17:40:07.135 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBhB+ZlBCB5BBAAcPdldNaWd2lMrlnbRN/P4A== 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.3] 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBkB+Zlc3eVCxAAPilyLSG+OX/mhnFg8OUliw== 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr.x] 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBXB+Zljt40MxAAMHpruRoL734TTSyfrrJ/Bw== 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.4] 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBqB+ZlscIcDBAAbC8/Rc0BhsSCqpVGfQc6RA== 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.5] 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBtB+ZloTNLERAArlCRCzKZjALYLcVfst92zQ== 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.6] 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBwB+ZlPNH8GBAAURssiBm3D/qerITPpfKr4A== 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.7] 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBzB+ZlbQAkIBAAc2pDk1VEHRmcpV3ZEmH8hQ== 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout:[client.0] 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBXB+Zle+D3NRAAuvGQwDami6GjsV3Uk2IKgg== 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout:[client.1] 2024-03-04T17:40:07.136 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBXB+ZlgJd7OBAAjaICGPAjpLEziTvz81jSpQ== 2024-03-04T17:40:07.137 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T17:40:07.173 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T17:40:07.224 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.227 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.311 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.313 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.392 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.397 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.475 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.489 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.554 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T17:40:07.573 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T17:40:07.639 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.653 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.724 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.737 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.775 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.817 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.861 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.897 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T17:40:07.945 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T17:40:07.977 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T17:40:08.029 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T17:40:08.061 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T17:40:08.117 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-04T17:40:08.117 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-04T17:40:08.154 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-04T17:40:08.247 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-04T17:40:08.301 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-04T17:40:08.352 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-04T17:40:08.442 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-04T17:40:08.452 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-04T17:40:08.466 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-04T17:40:08.558 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-04T17:40:08.568 DEBUG:teuthology.orchestra.run.smithi137:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:40:08.571 DEBUG:teuthology.orchestra.run.smithi173:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-04T17:40:08.616 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-04T17:40:08.617 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-04T17:40:08.617 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-04T17:40:08.619 INFO:tasks.ceph.mon.a:Started 2024-03-04T17:40:08.619 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-04T17:40:08.620 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-04T17:40:08.622 INFO:tasks.ceph.mon.c:Started 2024-03-04T17:40:08.622 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-04T17:40:08.622 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-04T17:40:08.661 INFO:tasks.ceph.mon.b:Started 2024-03-04T17:40:08.661 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-04T17:40:08.661 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-04T17:40:08.661 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-04T17:40:08.665 INFO:tasks.ceph.mgr.y:Started 2024-03-04T17:40:08.665 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-04T17:40:08.665 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-04T17:40:08.667 INFO:tasks.ceph.mgr.x:Started 2024-03-04T17:40:08.667 INFO:tasks.ceph:Setting crush tunables to default 2024-03-04T17:40:08.668 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd crush tunables default 2024-03-04T17:40:14.168 INFO:teuthology.orchestra.run.smithi137.stderr:adjusted tunables profile to default 2024-03-04T17:40:14.187 INFO:tasks.ceph:check_enable_crimson: False 2024-03-04T17:40:14.187 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-04T17:40:14.188 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:40:14.188 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-04T17:40:14.199 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:40:14.199 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-04T17:40:14.249 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:40:14.249 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-04T17:40:14.272 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:14.268+0000 7f346e7ab180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T17:40:14.284 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:40:14.284 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-04T17:40:14.334 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:14.332+0000 7f6a1a50f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T17:40:14.339 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:40:14.339 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-04T17:40:14.353 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:40:14.353 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-04T17:40:14.406 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:40:14.406 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-04T17:40:14.458 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-04T17:40:14.459 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-04T17:40:14.511 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 1beb1e6f-f79b-425b-8936-e75d85a48315 0 2024-03-04T17:40:14.742 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:14.736+0000 7f346e7ab180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T17:40:14.776 INFO:teuthology.orchestra.run.smithi173.stdout:0 2024-03-04T17:40:14.799 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 668353da-c867-4e02-9d2e-8846dfcda426 1 2024-03-04T17:40:14.864 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:14.864+0000 7f6a1a50f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T17:40:14.927 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:14.924+0000 7f346e7ab180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T17:40:15.007 INFO:teuthology.orchestra.run.smithi173.stdout:1 2024-03-04T17:40:15.024 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 7c387805-c1f7-465f-8fb8-cbcbc9ffd093 2 2024-03-04T17:40:15.101 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:15.100+0000 7f6a1a50f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T17:40:15.226 INFO:teuthology.orchestra.run.smithi173.stdout:2 2024-03-04T17:40:15.242 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 44bcf553-c776-4d69-9d06-77b228e52f03 3 2024-03-04T17:40:15.439 INFO:teuthology.orchestra.run.smithi173.stdout:3 2024-03-04T17:40:15.455 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 0c708501-d09d-48f1-a613-f7d7a9f8b19f 4 2024-03-04T17:40:15.472 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:15.468+0000 7f346e7ab180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T17:40:15.525 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:15.520+0000 7f346e7ab180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T17:40:15.577 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:15.572+0000 7f346e7ab180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T17:40:15.653 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:15.648+0000 7f346e7ab180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T17:40:15.661 INFO:teuthology.orchestra.run.smithi173.stdout:4 2024-03-04T17:40:15.673 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:15.672+0000 7f6a1a50f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T17:40:15.677 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new fea82914-802b-483c-aafb-302c37fe91b8 5 2024-03-04T17:40:15.705 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:15.700+0000 7f346e7ab180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T17:40:15.729 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:15.728+0000 7f6a1a50f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T17:40:15.781 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:15.776+0000 7f346e7ab180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T17:40:15.783 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:15.780+0000 7f6a1a50f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T17:40:15.834 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:15.828+0000 7f346e7ab180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T17:40:15.861 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:15.860+0000 7f6a1a50f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T17:40:15.872 INFO:teuthology.orchestra.run.smithi173.stdout:5 2024-03-04T17:40:15.888 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 0bd8bd8b-50d4-4ce6-aa49-2a00637f7fbb 6 2024-03-04T17:40:15.917 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:15.916+0000 7f6a1a50f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T17:40:15.982 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:15.976+0000 7f346e7ab180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T17:40:15.996 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:15.996+0000 7f6a1a50f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T17:40:16.044 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:16.040+0000 7f346e7ab180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T17:40:16.052 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:16.048+0000 7f6a1a50f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T17:40:16.079 INFO:teuthology.orchestra.run.smithi173.stdout:6 2024-03-04T17:40:16.096 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new ce8a7fb4-dbce-48d5-aed3-a85574f17e47 7 2024-03-04T17:40:16.205 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:16.204+0000 7f6a1a50f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T17:40:16.268 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:16.268+0000 7f6a1a50f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T17:40:16.343 INFO:teuthology.orchestra.run.smithi173.stdout:7 2024-03-04T17:40:16.360 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-04T17:40:16.360 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-04T17:40:16.362 INFO:tasks.ceph.osd.0:Started 2024-03-04T17:40:16.363 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-04T17:40:16.363 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-04T17:40:16.365 INFO:tasks.ceph.osd.1:Started 2024-03-04T17:40:16.365 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-04T17:40:16.365 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-04T17:40:16.367 INFO:tasks.ceph.osd.2:Started 2024-03-04T17:40:16.367 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-04T17:40:16.367 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-04T17:40:16.369 INFO:tasks.ceph.osd.3:Started 2024-03-04T17:40:16.369 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-04T17:40:16.370 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-04T17:40:16.372 INFO:tasks.ceph.osd.4:Started 2024-03-04T17:40:16.372 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-04T17:40:16.372 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-04T17:40:16.374 INFO:tasks.ceph.osd.5:Started 2024-03-04T17:40:16.374 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-04T17:40:16.374 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-04T17:40:16.376 INFO:tasks.ceph.osd.6:Started 2024-03-04T17:40:16.376 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-04T17:40:16.376 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-04T17:40:16.378 INFO:tasks.ceph.osd.7:Started 2024-03-04T17:40:16.378 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T17:40:16.593 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:40:16.594 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":10,"fsid":"4e7d04e9-67dd-413e-a1a3-7566f727539e","created":"2024-03-04T17:40:14.041298+0000","modified":"2024-03-04T17:40:16.336633+0000","last_up_change":"0.000000","last_in_change":"2024-03-04T17:40:16.336633+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1beb1e6f-f79b-425b-8936-e75d85a48315","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"668353da-c867-4e02-9d2e-8846dfcda426","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7c387805-c1f7-465f-8fb8-cbcbc9ffd093","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"44bcf553-c776-4d69-9d06-77b228e52f03","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0c708501-d09d-48f1-a613-f7d7a9f8b19f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fea82914-802b-483c-aafb-302c37fe91b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0bd8bd8b-50d4-4ce6-aa49-2a00637f7fbb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ce8a7fb4-dbce-48d5-aed3-a85574f17e47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T17:40:16.609 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-04T17:40:16.609 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-04T17:40:16.665 INFO:tasks.ceph.mgr.y.smithi137.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-04T17:40:16.666 INFO:tasks.ceph.mgr.y.smithi137.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-04T17:40:16.666 INFO:tasks.ceph.mgr.y.smithi137.stderr: from numpy import show_config as show_numpy_config 2024-03-04T17:40:16.710 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:16.704+0000 7f346e7ab180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T17:40:16.797 INFO:tasks.ceph.osd.0.smithi137.stderr:2024-03-04T17:40:16.792+0000 7fbd0240e7c0 -1 Falling back to public interface 2024-03-04T17:40:16.840 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:16.836+0000 7f346e7ab180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T17:40:16.972 INFO:tasks.ceph.mgr.x.smithi173.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-04T17:40:16.972 INFO:tasks.ceph.mgr.x.smithi173.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-04T17:40:16.972 INFO:tasks.ceph.mgr.x.smithi173.stderr: from numpy import show_config as show_numpy_config 2024-03-04T17:40:17.005 INFO:tasks.ceph.osd.5.smithi173.stderr:2024-03-04T17:40:17.004+0000 7f0171fa17c0 -1 Falling back to public interface 2024-03-04T17:40:17.006 INFO:tasks.ceph.osd.6.smithi173.stderr:2024-03-04T17:40:17.004+0000 7f21024897c0 -1 Falling back to public interface 2024-03-04T17:40:17.006 INFO:tasks.ceph.osd.7.smithi173.stderr:2024-03-04T17:40:17.004+0000 7fdbab04f7c0 -1 Falling back to public interface 2024-03-04T17:40:17.007 INFO:tasks.ceph.osd.4.smithi173.stderr:2024-03-04T17:40:17.004+0000 7f5b5ae047c0 -1 Falling back to public interface 2024-03-04T17:40:17.015 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:17.012+0000 7f6a1a50f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T17:40:17.016 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:17.012+0000 7f346e7ab180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T17:40:17.017 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-03-04T17:40:17.012+0000 7f44b17287c0 -1 Falling back to public interface 2024-03-04T17:40:17.018 INFO:tasks.ceph.osd.3.smithi137.stderr:2024-03-04T17:40:17.012+0000 7f9fb9b817c0 -1 Falling back to public interface 2024-03-04T17:40:17.032 INFO:tasks.ceph.osd.1.smithi137.stderr:2024-03-04T17:40:17.028+0000 7f08866a07c0 -1 Falling back to public interface 2024-03-04T17:40:17.083 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:17.076+0000 7f346e7ab180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T17:40:17.156 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:17.152+0000 7f6a1a50f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T17:40:17.215 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:17.208+0000 7f346e7ab180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T17:40:17.317 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:17.316+0000 7f6a1a50f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T17:40:17.380 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:17.380+0000 7f6a1a50f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T17:40:17.416 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-04T17:40:17.512 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:17.508+0000 7f6a1a50f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T17:40:17.590 INFO:teuthology.misc.health.smithi137.stdout: 2024-03-04T17:40:17.591 INFO:teuthology.misc.health.smithi137.stdout:{"epoch":10,"fsid":"4e7d04e9-67dd-413e-a1a3-7566f727539e","created":"2024-03-04T17:40:14.041298+0000","modified":"2024-03-04T17:40:16.336633+0000","last_up_change":"0.000000","last_in_change":"2024-03-04T17:40:16.336633+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1beb1e6f-f79b-425b-8936-e75d85a48315","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"668353da-c867-4e02-9d2e-8846dfcda426","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7c387805-c1f7-465f-8fb8-cbcbc9ffd093","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"44bcf553-c776-4d69-9d06-77b228e52f03","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0c708501-d09d-48f1-a613-f7d7a9f8b19f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fea82914-802b-483c-aafb-302c37fe91b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0bd8bd8b-50d4-4ce6-aa49-2a00637f7fbb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ce8a7fb4-dbce-48d5-aed3-a85574f17e47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T17:40:17.606 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-04T17:40:17.839 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:17.832+0000 7f346e7ab180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T17:40:17.912 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:17.908+0000 7f346e7ab180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T17:40:18.092 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:18.088+0000 7f346e7ab180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T17:40:18.124 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:18.124+0000 7f6a1a50f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T17:40:18.164 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:18.160+0000 7f346e7ab180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T17:40:18.199 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:18.196+0000 7f6a1a50f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T17:40:18.378 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:18.376+0000 7f6a1a50f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T17:40:18.448 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:18.444+0000 7f346e7ab180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T17:40:18.451 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:18.448+0000 7f6a1a50f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T17:40:18.509 INFO:tasks.ceph.mgr.y.smithi137.stderr:2024-03-04T17:40:18.504+0000 7f346e7ab180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T17:40:18.739 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:18.736+0000 7f6a1a50f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T17:40:18.801 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:18.800+0000 7f6a1a50f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T17:40:19.836 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:19.832+0000 7f69c1959640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-04T17:40:19.836 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-04T17:40:19.832+0000 7f69c1959640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-04T17:40:20.724 INFO:tasks.ceph.osd.0.smithi137.stderr:2024-03-04T17:40:20.720+0000 7fbd0240e7c0 -1 osd.0 0 log_to_monitors true 2024-03-04T17:40:20.792 INFO:tasks.ceph.osd.6.smithi173.stderr:2024-03-04T17:40:20.788+0000 7f21024897c0 -1 osd.6 0 log_to_monitors true 2024-03-04T17:40:20.882 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-03-04T17:40:20.876+0000 7f44b17287c0 -1 osd.2 0 log_to_monitors true 2024-03-04T17:40:20.898 INFO:tasks.ceph.osd.1.smithi137.stderr:2024-03-04T17:40:20.892+0000 7f08866a07c0 -1 osd.1 0 log_to_monitors true 2024-03-04T17:40:20.915 INFO:tasks.ceph.osd.3.smithi137.stderr:2024-03-04T17:40:20.908+0000 7f9fb9b817c0 -1 osd.3 0 log_to_monitors true 2024-03-04T17:40:20.920 INFO:tasks.ceph.osd.7.smithi173.stderr:2024-03-04T17:40:20.920+0000 7fdbab04f7c0 -1 osd.7 0 log_to_monitors true 2024-03-04T17:40:20.956 INFO:tasks.ceph.osd.4.smithi173.stderr:2024-03-04T17:40:20.956+0000 7f5b5ae047c0 -1 osd.4 0 log_to_monitors true 2024-03-04T17:40:20.967 INFO:tasks.ceph.osd.5.smithi173.stderr:2024-03-04T17:40:20.964+0000 7f0171fa17c0 -1 osd.5 0 log_to_monitors true 2024-03-04T17:40:22.766 INFO:tasks.ceph.osd.0.smithi137.stderr:2024-03-04T17:40:22.760+0000 7fbcff3ca640 -1 osd.0 0 waiting for initial osdmap 2024-03-04T17:40:22.778 INFO:tasks.ceph.osd.6.smithi173.stderr:2024-03-04T17:40:22.776+0000 7f20fdc0f640 -1 osd.6 0 waiting for initial osdmap 2024-03-04T17:40:22.785 INFO:tasks.ceph.osd.0.smithi137.stderr:2024-03-04T17:40:22.780+0000 7fbcf61db640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:40:22.809 INFO:tasks.ceph.osd.6.smithi173.stderr:2024-03-04T17:40:22.808+0000 7f20f6256640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:40:24.412 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-04T17:40:24.543 INFO:tasks.ceph.osd.5.smithi173.stderr:2024-03-04T17:40:24.540+0000 7f016ef5d640 -1 osd.5 0 waiting for initial osdmap 2024-03-04T17:40:24.557 INFO:tasks.ceph.osd.1.smithi137.stderr:2024-03-04T17:40:24.552+0000 7f0881e26640 -1 osd.1 0 waiting for initial osdmap 2024-03-04T17:40:24.565 INFO:tasks.ceph.osd.5.smithi173.stderr:2024-03-04T17:40:24.564+0000 7f0165d6e640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:40:24.590 INFO:tasks.ceph.osd.3.smithi137.stderr:2024-03-04T17:40:24.584+0000 7f9fb5307640 -1 osd.3 0 waiting for initial osdmap 2024-03-04T17:40:24.593 INFO:tasks.ceph.osd.4.smithi173.stderr:2024-03-04T17:40:24.592+0000 7f5b5658a640 -1 osd.4 0 waiting for initial osdmap 2024-03-04T17:40:24.600 INFO:tasks.ceph.osd.1.smithi137.stderr:2024-03-04T17:40:24.596+0000 7f087a46d640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:40:24.612 INFO:tasks.ceph.osd.7.smithi173.stderr:2024-03-04T17:40:24.612+0000 7fdba67d5640 -1 osd.7 0 waiting for initial osdmap 2024-03-04T17:40:24.615 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-03-04T17:40:24.608+0000 7f44aceae640 -1 osd.2 0 waiting for initial osdmap 2024-03-04T17:40:24.620 INFO:tasks.ceph.osd.3.smithi137.stderr:2024-03-04T17:40:24.616+0000 7f9fad94e640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:40:24.621 INFO:tasks.ceph.osd.4.smithi173.stderr:2024-03-04T17:40:24.620+0000 7f5b4ebd1640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:40:24.632 INFO:tasks.ceph.osd.7.smithi173.stderr:2024-03-04T17:40:24.632+0000 7fdb9ee1c640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:40:24.640 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-03-04T17:40:24.636+0000 7f44a54f5640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T17:40:24.753 INFO:teuthology.misc.health.smithi137.stdout: 2024-03-04T17:40:24.754 INFO:teuthology.misc.health.smithi137.stdout:{"epoch":14,"fsid":"4e7d04e9-67dd-413e-a1a3-7566f727539e","created":"2024-03-04T17:40:14.041298+0000","modified":"2024-03-04T17:40:23.884217+0000","last_up_change":"2024-03-04T17:40:22.844363+0000","last_in_change":"2024-03-04T17:40:16.336633+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T17:40:22.941292+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"1beb1e6f-f79b-425b-8936-e75d85a48315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6801","nonce":3095622620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6803","nonce":3095622620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6807","nonce":3095622620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6805","nonce":3095622620}]},"public_addr":"172.21.15.137:6801/3095622620","cluster_addr":"172.21.15.137:6803/3095622620","heartbeat_back_addr":"172.21.15.137:6807/3095622620","heartbeat_front_addr":"172.21.15.137:6805/3095622620","state":["exists","up"]},{"osd":1,"uuid":"668353da-c867-4e02-9d2e-8846dfcda426","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7c387805-c1f7-465f-8fb8-cbcbc9ffd093","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"44bcf553-c776-4d69-9d06-77b228e52f03","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0c708501-d09d-48f1-a613-f7d7a9f8b19f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fea82914-802b-483c-aafb-302c37fe91b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0bd8bd8b-50d4-4ce6-aa49-2a00637f7fbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6810","nonce":2740311098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6818","nonce":2740311098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6831","nonce":2740311098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6825","nonce":2740311098}]},"public_addr":"172.21.15.173:6810/2740311098","cluster_addr":"172.21.15.173:6818/2740311098","heartbeat_back_addr":"172.21.15.173:6831/2740311098","heartbeat_front_addr":"172.21.15.173:6825/2740311098","state":["exists","up"]},{"osd":7,"uuid":"ce8a7fb4-dbce-48d5-aed3-a85574f17e47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.744772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.838580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T17:40:24.772 DEBUG:teuthology.misc:2 of 8 OSDs are up 2024-03-04T17:40:31.577 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-04T17:40:31.873 INFO:teuthology.misc.health.smithi137.stdout: 2024-03-04T17:40:31.873 INFO:teuthology.misc.health.smithi137.stdout:{"epoch":17,"fsid":"4e7d04e9-67dd-413e-a1a3-7566f727539e","created":"2024-03-04T17:40:14.041298+0000","modified":"2024-03-04T17:40:26.966722+0000","last_up_change":"2024-03-04T17:40:24.946074+0000","last_in_change":"2024-03-04T17:40:16.336633+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T17:40:22.941292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1beb1e6f-f79b-425b-8936-e75d85a48315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6801","nonce":3095622620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6803","nonce":3095622620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6807","nonce":3095622620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6805","nonce":3095622620}]},"public_addr":"172.21.15.137:6801/3095622620","cluster_addr":"172.21.15.137:6803/3095622620","heartbeat_back_addr":"172.21.15.137:6807/3095622620","heartbeat_front_addr":"172.21.15.137:6805/3095622620","state":["exists","up"]},{"osd":1,"uuid":"668353da-c867-4e02-9d2e-8846dfcda426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":1039931256},{"type":"v1","addr":"172.21.15.137:6825","nonce":1039931256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":1039931256},{"type":"v1","addr":"172.21.15.137:6827","nonce":1039931256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":1039931256},{"type":"v1","addr":"172.21.15.137:6831","nonce":1039931256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":1039931256},{"type":"v1","addr":"172.21.15.137:6829","nonce":1039931256}]},"public_addr":"172.21.15.137:6825/1039931256","cluster_addr":"172.21.15.137:6827/1039931256","heartbeat_back_addr":"172.21.15.137:6831/1039931256","heartbeat_front_addr":"172.21.15.137:6829/1039931256","state":["exists","up"]},{"osd":2,"uuid":"7c387805-c1f7-465f-8fb8-cbcbc9ffd093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3491386063},{"type":"v1","addr":"172.21.15.137:6810","nonce":3491386063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3491386063},{"type":"v1","addr":"172.21.15.137:6815","nonce":3491386063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3491386063},{"type":"v1","addr":"172.21.15.137:6823","nonce":3491386063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6817","nonce":3491386063},{"type":"v1","addr":"172.21.15.137:6820","nonce":3491386063}]},"public_addr":"172.21.15.137:6810/3491386063","cluster_addr":"172.21.15.137:6815/3491386063","heartbeat_back_addr":"172.21.15.137:6823/3491386063","heartbeat_front_addr":"172.21.15.137:6820/3491386063","state":["exists","up"]},{"osd":3,"uuid":"44bcf553-c776-4d69-9d06-77b228e52f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":3854517010},{"type":"v1","addr":"172.21.15.137:6811","nonce":3854517010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":3854517010},{"type":"v1","addr":"172.21.15.137:6814","nonce":3854517010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":3854517010},{"type":"v1","addr":"172.21.15.137:6821","nonce":3854517010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3854517010},{"type":"v1","addr":"172.21.15.137:6818","nonce":3854517010}]},"public_addr":"172.21.15.137:6811/3854517010","cluster_addr":"172.21.15.137:6814/3854517010","heartbeat_back_addr":"172.21.15.137:6821/3854517010","heartbeat_front_addr":"172.21.15.137:6818/3854517010","state":["exists","up"]},{"osd":4,"uuid":"0c708501-d09d-48f1-a613-f7d7a9f8b19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6809","nonce":4214690098},{"type":"v1","addr":"172.21.15.173:6813","nonce":4214690098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6817","nonce":4214690098},{"type":"v1","addr":"172.21.15.173:6820","nonce":4214690098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6829","nonce":4214690098},{"type":"v1","addr":"172.21.15.173:6830","nonce":4214690098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":4214690098},{"type":"v1","addr":"172.21.15.173:6827","nonce":4214690098}]},"public_addr":"172.21.15.173:6813/4214690098","cluster_addr":"172.21.15.173:6820/4214690098","heartbeat_back_addr":"172.21.15.173:6830/4214690098","heartbeat_front_addr":"172.21.15.173:6827/4214690098","state":["exists","up"]},{"osd":5,"uuid":"fea82914-802b-483c-aafb-302c37fe91b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3687068977},{"type":"v1","addr":"172.21.15.173:6801","nonce":3687068977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3687068977},{"type":"v1","addr":"172.21.15.173:6803","nonce":3687068977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6811","nonce":3687068977},{"type":"v1","addr":"172.21.15.173:6816","nonce":3687068977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3687068977},{"type":"v1","addr":"172.21.15.173:6807","nonce":3687068977}]},"public_addr":"172.21.15.173:6801/3687068977","cluster_addr":"172.21.15.173:6803/3687068977","heartbeat_back_addr":"172.21.15.173:6816/3687068977","heartbeat_front_addr":"172.21.15.173:6807/3687068977","state":["exists","up"]},{"osd":6,"uuid":"0bd8bd8b-50d4-4ce6-aa49-2a00637f7fbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6810","nonce":2740311098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6818","nonce":2740311098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6831","nonce":2740311098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6825","nonce":2740311098}]},"public_addr":"172.21.15.173:6810/2740311098","cluster_addr":"172.21.15.173:6818/2740311098","heartbeat_back_addr":"172.21.15.173:6831/2740311098","heartbeat_front_addr":"172.21.15.173:6825/2740311098","state":["exists","up"]},{"osd":7,"uuid":"ce8a7fb4-dbce-48d5-aed3-a85574f17e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6805","nonce":879938785},{"type":"v1","addr":"172.21.15.173:6808","nonce":879938785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":879938785},{"type":"v1","addr":"172.21.15.173:6815","nonce":879938785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":879938785},{"type":"v1","addr":"172.21.15.173:6826","nonce":879938785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":879938785},{"type":"v1","addr":"172.21.15.173:6821","nonce":879938785}]},"public_addr":"172.21.15.173:6808/879938785","cluster_addr":"172.21.15.173:6815/879938785","heartbeat_back_addr":"172.21.15.173:6826/879938785","heartbeat_front_addr":"172.21.15.173:6821/879938785","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.744772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.855401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.896004+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.892247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.919350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.971018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.838580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.877920+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T17:40:31.892 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-04T17:40:31.892 INFO:tasks.ceph:Creating RBD pool 2024-03-04T17:40:31.892 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd pool create rbd 8 2024-03-04T17:40:33.000 INFO:teuthology.orchestra.run.smithi137.stderr:pool 'rbd' created 2024-03-04T17:40:33.024 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-03-04T17:40:34.003 INFO:teuthology.orchestra.run.smithi137.stderr:enabled application 'rbd' on pool 'rbd' 2024-03-04T17:40:34.029 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-04T17:40:34.029 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-04T17:40:34.029 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-04T17:40:34.029 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-04T17:40:34.030 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-04T17:40:34.375 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:40:34.393 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":6,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":3535769218},{"type":"v1","addr":"172.21.15.173:6833","nonce":3535769218}]},"active_addr":"172.21.15.173:6833/3535769218","active_change":"2024-03-04T17:40:18.823358+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":2779436414}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":570188723}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":1133394148}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":1671218274}]}]} 2024-03-04T17:40:34.396 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-04T17:40:34.396 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-04T17:40:34.397 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T17:40:34.691 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:40:34.692 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":19,"fsid":"4e7d04e9-67dd-413e-a1a3-7566f727539e","created":"2024-03-04T17:40:14.041298+0000","modified":"2024-03-04T17:40:33.995253+0000","last_up_change":"2024-03-04T17:40:24.946074+0000","last_in_change":"2024-03-04T17:40:16.336633+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T17:40:22.941292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-04T17:40:32.185090+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1beb1e6f-f79b-425b-8936-e75d85a48315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6801","nonce":3095622620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6803","nonce":3095622620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6807","nonce":3095622620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6805","nonce":3095622620}]},"public_addr":"172.21.15.137:6801/3095622620","cluster_addr":"172.21.15.137:6803/3095622620","heartbeat_back_addr":"172.21.15.137:6807/3095622620","heartbeat_front_addr":"172.21.15.137:6805/3095622620","state":["exists","up"]},{"osd":1,"uuid":"668353da-c867-4e02-9d2e-8846dfcda426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":1039931256},{"type":"v1","addr":"172.21.15.137:6825","nonce":1039931256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":1039931256},{"type":"v1","addr":"172.21.15.137:6827","nonce":1039931256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":1039931256},{"type":"v1","addr":"172.21.15.137:6831","nonce":1039931256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":1039931256},{"type":"v1","addr":"172.21.15.137:6829","nonce":1039931256}]},"public_addr":"172.21.15.137:6825/1039931256","cluster_addr":"172.21.15.137:6827/1039931256","heartbeat_back_addr":"172.21.15.137:6831/1039931256","heartbeat_front_addr":"172.21.15.137:6829/1039931256","state":["exists","up"]},{"osd":2,"uuid":"7c387805-c1f7-465f-8fb8-cbcbc9ffd093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3491386063},{"type":"v1","addr":"172.21.15.137:6810","nonce":3491386063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3491386063},{"type":"v1","addr":"172.21.15.137:6815","nonce":3491386063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3491386063},{"type":"v1","addr":"172.21.15.137:6823","nonce":3491386063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6817","nonce":3491386063},{"type":"v1","addr":"172.21.15.137:6820","nonce":3491386063}]},"public_addr":"172.21.15.137:6810/3491386063","cluster_addr":"172.21.15.137:6815/3491386063","heartbeat_back_addr":"172.21.15.137:6823/3491386063","heartbeat_front_addr":"172.21.15.137:6820/3491386063","state":["exists","up"]},{"osd":3,"uuid":"44bcf553-c776-4d69-9d06-77b228e52f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":3854517010},{"type":"v1","addr":"172.21.15.137:6811","nonce":3854517010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":3854517010},{"type":"v1","addr":"172.21.15.137:6814","nonce":3854517010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":3854517010},{"type":"v1","addr":"172.21.15.137:6821","nonce":3854517010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3854517010},{"type":"v1","addr":"172.21.15.137:6818","nonce":3854517010}]},"public_addr":"172.21.15.137:6811/3854517010","cluster_addr":"172.21.15.137:6814/3854517010","heartbeat_back_addr":"172.21.15.137:6821/3854517010","heartbeat_front_addr":"172.21.15.137:6818/3854517010","state":["exists","up"]},{"osd":4,"uuid":"0c708501-d09d-48f1-a613-f7d7a9f8b19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6809","nonce":4214690098},{"type":"v1","addr":"172.21.15.173:6813","nonce":4214690098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6817","nonce":4214690098},{"type":"v1","addr":"172.21.15.173:6820","nonce":4214690098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6829","nonce":4214690098},{"type":"v1","addr":"172.21.15.173:6830","nonce":4214690098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":4214690098},{"type":"v1","addr":"172.21.15.173:6827","nonce":4214690098}]},"public_addr":"172.21.15.173:6813/4214690098","cluster_addr":"172.21.15.173:6820/4214690098","heartbeat_back_addr":"172.21.15.173:6830/4214690098","heartbeat_front_addr":"172.21.15.173:6827/4214690098","state":["exists","up"]},{"osd":5,"uuid":"fea82914-802b-483c-aafb-302c37fe91b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3687068977},{"type":"v1","addr":"172.21.15.173:6801","nonce":3687068977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3687068977},{"type":"v1","addr":"172.21.15.173:6803","nonce":3687068977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6811","nonce":3687068977},{"type":"v1","addr":"172.21.15.173:6816","nonce":3687068977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3687068977},{"type":"v1","addr":"172.21.15.173:6807","nonce":3687068977}]},"public_addr":"172.21.15.173:6801/3687068977","cluster_addr":"172.21.15.173:6803/3687068977","heartbeat_back_addr":"172.21.15.173:6816/3687068977","heartbeat_front_addr":"172.21.15.173:6807/3687068977","state":["exists","up"]},{"osd":6,"uuid":"0bd8bd8b-50d4-4ce6-aa49-2a00637f7fbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6810","nonce":2740311098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6818","nonce":2740311098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6831","nonce":2740311098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6825","nonce":2740311098}]},"public_addr":"172.21.15.173:6810/2740311098","cluster_addr":"172.21.15.173:6818/2740311098","heartbeat_back_addr":"172.21.15.173:6831/2740311098","heartbeat_front_addr":"172.21.15.173:6825/2740311098","state":["exists","up"]},{"osd":7,"uuid":"ce8a7fb4-dbce-48d5-aed3-a85574f17e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6805","nonce":879938785},{"type":"v1","addr":"172.21.15.173:6808","nonce":879938785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":879938785},{"type":"v1","addr":"172.21.15.173:6815","nonce":879938785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":879938785},{"type":"v1","addr":"172.21.15.173:6826","nonce":879938785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":879938785},{"type":"v1","addr":"172.21.15.173:6821","nonce":879938785}]},"public_addr":"172.21.15.173:6808/879938785","cluster_addr":"172.21.15.173:6815/879938785","heartbeat_back_addr":"172.21.15.173:6826/879938785","heartbeat_front_addr":"172.21.15.173:6821/879938785","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.744772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.855401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.896004+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.892247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.919350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.971018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.838580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.877920+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T17:40:34.710 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-04T17:40:34.711 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T17:40:34.997 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:40:34.998 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":19,"fsid":"4e7d04e9-67dd-413e-a1a3-7566f727539e","created":"2024-03-04T17:40:14.041298+0000","modified":"2024-03-04T17:40:33.995253+0000","last_up_change":"2024-03-04T17:40:24.946074+0000","last_in_change":"2024-03-04T17:40:16.336633+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T17:40:22.941292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-04T17:40:32.185090+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1beb1e6f-f79b-425b-8936-e75d85a48315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6801","nonce":3095622620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6803","nonce":3095622620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6807","nonce":3095622620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3095622620},{"type":"v1","addr":"172.21.15.137:6805","nonce":3095622620}]},"public_addr":"172.21.15.137:6801/3095622620","cluster_addr":"172.21.15.137:6803/3095622620","heartbeat_back_addr":"172.21.15.137:6807/3095622620","heartbeat_front_addr":"172.21.15.137:6805/3095622620","state":["exists","up"]},{"osd":1,"uuid":"668353da-c867-4e02-9d2e-8846dfcda426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":1039931256},{"type":"v1","addr":"172.21.15.137:6825","nonce":1039931256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":1039931256},{"type":"v1","addr":"172.21.15.137:6827","nonce":1039931256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":1039931256},{"type":"v1","addr":"172.21.15.137:6831","nonce":1039931256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":1039931256},{"type":"v1","addr":"172.21.15.137:6829","nonce":1039931256}]},"public_addr":"172.21.15.137:6825/1039931256","cluster_addr":"172.21.15.137:6827/1039931256","heartbeat_back_addr":"172.21.15.137:6831/1039931256","heartbeat_front_addr":"172.21.15.137:6829/1039931256","state":["exists","up"]},{"osd":2,"uuid":"7c387805-c1f7-465f-8fb8-cbcbc9ffd093","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3491386063},{"type":"v1","addr":"172.21.15.137:6810","nonce":3491386063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3491386063},{"type":"v1","addr":"172.21.15.137:6815","nonce":3491386063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3491386063},{"type":"v1","addr":"172.21.15.137:6823","nonce":3491386063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6817","nonce":3491386063},{"type":"v1","addr":"172.21.15.137:6820","nonce":3491386063}]},"public_addr":"172.21.15.137:6810/3491386063","cluster_addr":"172.21.15.137:6815/3491386063","heartbeat_back_addr":"172.21.15.137:6823/3491386063","heartbeat_front_addr":"172.21.15.137:6820/3491386063","state":["exists","up"]},{"osd":3,"uuid":"44bcf553-c776-4d69-9d06-77b228e52f03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":3854517010},{"type":"v1","addr":"172.21.15.137:6811","nonce":3854517010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":3854517010},{"type":"v1","addr":"172.21.15.137:6814","nonce":3854517010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6819","nonce":3854517010},{"type":"v1","addr":"172.21.15.137:6821","nonce":3854517010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3854517010},{"type":"v1","addr":"172.21.15.137:6818","nonce":3854517010}]},"public_addr":"172.21.15.137:6811/3854517010","cluster_addr":"172.21.15.137:6814/3854517010","heartbeat_back_addr":"172.21.15.137:6821/3854517010","heartbeat_front_addr":"172.21.15.137:6818/3854517010","state":["exists","up"]},{"osd":4,"uuid":"0c708501-d09d-48f1-a613-f7d7a9f8b19f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6809","nonce":4214690098},{"type":"v1","addr":"172.21.15.173:6813","nonce":4214690098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6817","nonce":4214690098},{"type":"v1","addr":"172.21.15.173:6820","nonce":4214690098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6829","nonce":4214690098},{"type":"v1","addr":"172.21.15.173:6830","nonce":4214690098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":4214690098},{"type":"v1","addr":"172.21.15.173:6827","nonce":4214690098}]},"public_addr":"172.21.15.173:6813/4214690098","cluster_addr":"172.21.15.173:6820/4214690098","heartbeat_back_addr":"172.21.15.173:6830/4214690098","heartbeat_front_addr":"172.21.15.173:6827/4214690098","state":["exists","up"]},{"osd":5,"uuid":"fea82914-802b-483c-aafb-302c37fe91b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3687068977},{"type":"v1","addr":"172.21.15.173:6801","nonce":3687068977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3687068977},{"type":"v1","addr":"172.21.15.173:6803","nonce":3687068977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6811","nonce":3687068977},{"type":"v1","addr":"172.21.15.173:6816","nonce":3687068977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3687068977},{"type":"v1","addr":"172.21.15.173:6807","nonce":3687068977}]},"public_addr":"172.21.15.173:6801/3687068977","cluster_addr":"172.21.15.173:6803/3687068977","heartbeat_back_addr":"172.21.15.173:6816/3687068977","heartbeat_front_addr":"172.21.15.173:6807/3687068977","state":["exists","up"]},{"osd":6,"uuid":"0bd8bd8b-50d4-4ce6-aa49-2a00637f7fbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6810","nonce":2740311098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6818","nonce":2740311098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6831","nonce":2740311098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":2740311098},{"type":"v1","addr":"172.21.15.173:6825","nonce":2740311098}]},"public_addr":"172.21.15.173:6810/2740311098","cluster_addr":"172.21.15.173:6818/2740311098","heartbeat_back_addr":"172.21.15.173:6831/2740311098","heartbeat_front_addr":"172.21.15.173:6825/2740311098","state":["exists","up"]},{"osd":7,"uuid":"ce8a7fb4-dbce-48d5-aed3-a85574f17e47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6805","nonce":879938785},{"type":"v1","addr":"172.21.15.173:6808","nonce":879938785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":879938785},{"type":"v1","addr":"172.21.15.173:6815","nonce":879938785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":879938785},{"type":"v1","addr":"172.21.15.173:6826","nonce":879938785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":879938785},{"type":"v1","addr":"172.21.15.173:6821","nonce":879938785}]},"public_addr":"172.21.15.173:6808/879938785","cluster_addr":"172.21.15.173:6815/879938785","heartbeat_back_addr":"172.21.15.173:6826/879938785","heartbeat_front_addr":"172.21.15.173:6821/879938785","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.744772+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.855401+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.896004+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.892247+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.919350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.971018+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.838580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T17:40:21.877920+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T17:40:35.019 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-04T17:40:35.019 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-04T17:40:35.020 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-04T17:40:35.020 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-04T17:40:35.020 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-04T17:40:35.021 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-04T17:40:35.021 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-04T17:40:35.022 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-04T17:40:35.263 INFO:teuthology.orchestra.run.smithi137.stdout:64424509445 2024-03-04T17:40:35.263 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-04T17:40:35.286 INFO:teuthology.orchestra.run.smithi137.stdout:64424509445 2024-03-04T17:40:35.286 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-04T17:40:35.288 INFO:teuthology.orchestra.run.smithi137.stdout:64424509446 2024-03-04T17:40:35.288 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-04T17:40:35.290 INFO:teuthology.orchestra.run.smithi137.stdout:55834574853 2024-03-04T17:40:35.290 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-04T17:40:35.292 INFO:teuthology.orchestra.run.smithi137.stdout:64424509446 2024-03-04T17:40:35.292 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-04T17:40:35.300 INFO:teuthology.orchestra.run.smithi137.stdout:64424509445 2024-03-04T17:40:35.300 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-04T17:40:35.306 INFO:teuthology.orchestra.run.smithi137.stdout:64424509445 2024-03-04T17:40:35.306 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-04T17:40:35.314 INFO:teuthology.orchestra.run.smithi137.stdout:55834574854 2024-03-04T17:40:35.315 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-04T17:40:35.772 INFO:teuthology.orchestra.run.smithi137.stdout:64424509444 2024-03-04T17:40:35.780 INFO:teuthology.orchestra.run.smithi137.stdout:64424509444 2024-03-04T17:40:35.788 INFO:teuthology.orchestra.run.smithi137.stdout:64424509445 2024-03-04T17:40:35.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.3 2024-03-04T17:40:35.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.1 2024-03-04T17:40:35.815 INFO:teuthology.orchestra.run.smithi137.stdout:55834574852 2024-03-04T17:40:35.818 INFO:teuthology.orchestra.run.smithi137.stdout:64424509444 2024-03-04T17:40:35.822 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2024-03-04T17:40:35.825 INFO:teuthology.orchestra.run.smithi137.stdout:64424509445 2024-03-04T17:40:35.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-03-04T17:40:35.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.2 2024-03-04T17:40:35.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.5 2024-03-04T17:40:35.850 INFO:teuthology.orchestra.run.smithi137.stdout:55834574853 2024-03-04T17:40:35.856 INFO:teuthology.orchestra.run.smithi137.stdout:64424509444 2024-03-04T17:40:35.870 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-03-04T17:40:35.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.7 2024-03-04T17:40:36.799 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-04T17:40:36.811 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-04T17:40:36.823 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-04T17:40:36.839 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-04T17:40:36.847 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-04T17:40:36.848 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-04T17:40:36.870 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-04T17:40:36.876 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-04T17:40:37.272 INFO:teuthology.orchestra.run.smithi137.stdout:64424509446 2024-03-04T17:40:37.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.3 2024-03-04T17:40:37.302 DEBUG:teuthology.parallel:result is None 2024-03-04T17:40:37.322 INFO:teuthology.orchestra.run.smithi137.stdout:64424509447 2024-03-04T17:40:37.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.2 2024-03-04T17:40:37.347 DEBUG:teuthology.parallel:result is None 2024-03-04T17:40:37.351 INFO:teuthology.orchestra.run.smithi137.stdout:64424509447 2024-03-04T17:40:37.361 INFO:teuthology.orchestra.run.smithi137.stdout:64424509446 2024-03-04T17:40:37.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.4 2024-03-04T17:40:37.370 DEBUG:teuthology.parallel:result is None 2024-03-04T17:40:37.379 INFO:teuthology.orchestra.run.smithi137.stdout:55834574853 2024-03-04T17:40:37.380 INFO:teuthology.orchestra.run.smithi137.stdout:64424509446 2024-03-04T17:40:37.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.1 2024-03-04T17:40:37.386 DEBUG:teuthology.parallel:result is None 2024-03-04T17:40:37.400 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.5 2024-03-04T17:40:37.400 DEBUG:teuthology.parallel:result is None 2024-03-04T17:40:37.407 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-03-04T17:40:37.407 DEBUG:teuthology.parallel:result is None 2024-03-04T17:40:37.410 INFO:teuthology.orchestra.run.smithi137.stdout:64424509446 2024-03-04T17:40:37.421 INFO:teuthology.orchestra.run.smithi137.stdout:55834574854 2024-03-04T17:40:37.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.7 2024-03-04T17:40:37.427 DEBUG:teuthology.parallel:result is None 2024-03-04T17:40:37.439 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-03-04T17:40:37.439 DEBUG:teuthology.parallel:result is None 2024-03-04T17:40:37.439 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-04T17:40:37.439 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T17:40:37.746 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:40:37.746 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-04T17:40:37.762 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-03-04T17:40:36.835445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218816,"kb_used_data":2624,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755904,"statfs":{"total":773094113280,"available":772870045696,"internally_reserved":0,"allocated":2686976,"data_stored":1668288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001181"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:40:34.110600+0000","last_change":"2024-03-04T17:40:34.108385+0000","last_active":"2024-03-04T17:40:34.110600+0000","last_peered":"2024-03-04T17:40:34.110600+0000","last_clean":"2024-03-04T17:40:34.110600+0000","last_became_active":"2024-03-04T17:40:34.107996+0000","last_became_peered":"2024-03-04T17:40:34.107996+0000","last_unstale":"2024-03-04T17:40:34.110600+0000","last_undegraded":"2024-03-04T17:40:34.110600+0000","last_fullsized":"2024-03-04T17:40:34.110600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:38:40.225957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:40:34.110510+0000","last_change":"2024-03-04T17:40:34.109516+0000","last_active":"2024-03-04T17:40:34.110510+0000","last_peered":"2024-03-04T17:40:34.110510+0000","last_clean":"2024-03-04T17:40:34.110510+0000","last_became_active":"2024-03-04T17:40:34.108801+0000","last_became_peered":"2024-03-04T17:40:34.108801+0000","last_unstale":"2024-03-04T17:40:34.110510+0000","last_undegraded":"2024-03-04T17:40:34.110510+0000","last_fullsized":"2024-03-04T17:40:34.110510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T04:18:11.770984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:40:34.013207+0000","last_change":"2024-03-04T17:40:34.012206+0000","last_active":"2024-03-04T17:40:34.013207+0000","last_peered":"2024-03-04T17:40:34.013207+0000","last_clean":"2024-03-04T17:40:34.013207+0000","last_became_active":"2024-03-04T17:40:34.011878+0000","last_became_peered":"2024-03-04T17:40:34.011878+0000","last_unstale":"2024-03-04T17:40:34.013207+0000","last_undegraded":"2024-03-04T17:40:34.013207+0000","last_fullsized":"2024-03-04T17:40:34.013207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:19:09.020311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T17:40:35.017983+0000","last_change":"2024-03-04T17:40:34.015957+0000","last_active":"2024-03-04T17:40:35.017983+0000","last_peered":"2024-03-04T17:40:35.017983+0000","last_clean":"2024-03-04T17:40:35.017983+0000","last_became_active":"2024-03-04T17:40:34.014438+0000","last_became_peered":"2024-03-04T17:40:34.014438+0000","last_unstale":"2024-03-04T17:40:35.017983+0000","last_undegraded":"2024-03-04T17:40:35.017983+0000","last_fullsized":"2024-03-04T17:40:35.017983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:22:27.779080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:40:34.012455+0000","last_change":"2024-03-04T17:40:34.011269+0000","last_active":"2024-03-04T17:40:34.012455+0000","last_peered":"2024-03-04T17:40:34.012455+0000","last_clean":"2024-03-04T17:40:34.012455+0000","last_became_active":"2024-03-04T17:40:34.010896+0000","last_became_peered":"2024-03-04T17:40:34.010896+0000","last_unstale":"2024-03-04T17:40:34.012455+0000","last_undegraded":"2024-03-04T17:40:34.012455+0000","last_fullsized":"2024-03-04T17:40:34.012455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T04:44:07.056232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:40:34.018555+0000","last_change":"2024-03-04T17:40:34.015732+0000","last_active":"2024-03-04T17:40:34.018555+0000","last_peered":"2024-03-04T17:40:34.018555+0000","last_clean":"2024-03-04T17:40:34.018555+0000","last_became_active":"2024-03-04T17:40:34.014871+0000","last_became_peered":"2024-03-04T17:40:34.014871+0000","last_unstale":"2024-03-04T17:40:34.018555+0000","last_undegraded":"2024-03-04T17:40:34.018555+0000","last_fullsized":"2024-03-04T17:40:34.018555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T04:24:41.029343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:40:34.016711+0000","last_change":"2024-03-04T17:40:34.013700+0000","last_active":"2024-03-04T17:40:34.016711+0000","last_peered":"2024-03-04T17:40:34.016711+0000","last_clean":"2024-03-04T17:40:34.016711+0000","last_became_active":"2024-03-04T17:40:34.013287+0000","last_became_peered":"2024-03-04T17:40:34.013287+0000","last_unstale":"2024-03-04T17:40:34.016711+0000","last_undegraded":"2024-03-04T17:40:34.016711+0000","last_fullsized":"2024-03-04T17:40:34.016711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:59:38.174740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:40:34.011011+0000","last_change":"2024-03-04T17:40:34.010119+0000","last_active":"2024-03-04T17:40:34.011011+0000","last_peered":"2024-03-04T17:40:34.011011+0000","last_clean":"2024-03-04T17:40:34.011011+0000","last_became_active":"2024-03-04T17:40:34.009636+0000","last_became_peered":"2024-03-04T17:40:34.009636+0000","last_unstale":"2024-03-04T17:40:34.011011+0000","last_undegraded":"2024-03-04T17:40:34.011011+0000","last_fullsized":"2024-03-04T17:40:34.011011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:11:58.354928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T17:40:35.019441+0000","last_change":"2024-03-04T17:40:26.000245+0000","last_active":"2024-03-04T17:40:35.019441+0000","last_peered":"2024-03-04T17:40:35.019441+0000","last_clean":"2024-03-04T17:40:35.019441+0000","last_became_active":"2024-03-04T17:40:25.998187+0000","last_became_peered":"2024-03-04T17:40:25.998187+0000","last_unstale":"2024-03-04T17:40:35.019441+0000","last_undegraded":"2024-03-04T17:40:35.019441+0000","last_fullsized":"2024-03-04T17:40:35.019441+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:23.884217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:23.884217+0000","last_clean_scrub_stamp":"2024-03-04T17:40:23.884217+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:17:30.523252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":59507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":655623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":59507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":59507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":59507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":655623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":59507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":59507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T17:40:37.763 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T17:40:38.030 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:40:38.031 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-04T17:40:38.047 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-03-04T17:40:36.835445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218816,"kb_used_data":2624,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755904,"statfs":{"total":773094113280,"available":772870045696,"internally_reserved":0,"allocated":2686976,"data_stored":1668288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001181"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:40:34.110600+0000","last_change":"2024-03-04T17:40:34.108385+0000","last_active":"2024-03-04T17:40:34.110600+0000","last_peered":"2024-03-04T17:40:34.110600+0000","last_clean":"2024-03-04T17:40:34.110600+0000","last_became_active":"2024-03-04T17:40:34.107996+0000","last_became_peered":"2024-03-04T17:40:34.107996+0000","last_unstale":"2024-03-04T17:40:34.110600+0000","last_undegraded":"2024-03-04T17:40:34.110600+0000","last_fullsized":"2024-03-04T17:40:34.110600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:38:40.225957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:40:34.110510+0000","last_change":"2024-03-04T17:40:34.109516+0000","last_active":"2024-03-04T17:40:34.110510+0000","last_peered":"2024-03-04T17:40:34.110510+0000","last_clean":"2024-03-04T17:40:34.110510+0000","last_became_active":"2024-03-04T17:40:34.108801+0000","last_became_peered":"2024-03-04T17:40:34.108801+0000","last_unstale":"2024-03-04T17:40:34.110510+0000","last_undegraded":"2024-03-04T17:40:34.110510+0000","last_fullsized":"2024-03-04T17:40:34.110510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T04:18:11.770984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:40:34.013207+0000","last_change":"2024-03-04T17:40:34.012206+0000","last_active":"2024-03-04T17:40:34.013207+0000","last_peered":"2024-03-04T17:40:34.013207+0000","last_clean":"2024-03-04T17:40:34.013207+0000","last_became_active":"2024-03-04T17:40:34.011878+0000","last_became_peered":"2024-03-04T17:40:34.011878+0000","last_unstale":"2024-03-04T17:40:34.013207+0000","last_undegraded":"2024-03-04T17:40:34.013207+0000","last_fullsized":"2024-03-04T17:40:34.013207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:19:09.020311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T17:40:35.017983+0000","last_change":"2024-03-04T17:40:34.015957+0000","last_active":"2024-03-04T17:40:35.017983+0000","last_peered":"2024-03-04T17:40:35.017983+0000","last_clean":"2024-03-04T17:40:35.017983+0000","last_became_active":"2024-03-04T17:40:34.014438+0000","last_became_peered":"2024-03-04T17:40:34.014438+0000","last_unstale":"2024-03-04T17:40:35.017983+0000","last_undegraded":"2024-03-04T17:40:35.017983+0000","last_fullsized":"2024-03-04T17:40:35.017983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:22:27.779080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:40:34.012455+0000","last_change":"2024-03-04T17:40:34.011269+0000","last_active":"2024-03-04T17:40:34.012455+0000","last_peered":"2024-03-04T17:40:34.012455+0000","last_clean":"2024-03-04T17:40:34.012455+0000","last_became_active":"2024-03-04T17:40:34.010896+0000","last_became_peered":"2024-03-04T17:40:34.010896+0000","last_unstale":"2024-03-04T17:40:34.012455+0000","last_undegraded":"2024-03-04T17:40:34.012455+0000","last_fullsized":"2024-03-04T17:40:34.012455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T04:44:07.056232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:40:34.018555+0000","last_change":"2024-03-04T17:40:34.015732+0000","last_active":"2024-03-04T17:40:34.018555+0000","last_peered":"2024-03-04T17:40:34.018555+0000","last_clean":"2024-03-04T17:40:34.018555+0000","last_became_active":"2024-03-04T17:40:34.014871+0000","last_became_peered":"2024-03-04T17:40:34.014871+0000","last_unstale":"2024-03-04T17:40:34.018555+0000","last_undegraded":"2024-03-04T17:40:34.018555+0000","last_fullsized":"2024-03-04T17:40:34.018555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T04:24:41.029343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:40:34.016711+0000","last_change":"2024-03-04T17:40:34.013700+0000","last_active":"2024-03-04T17:40:34.016711+0000","last_peered":"2024-03-04T17:40:34.016711+0000","last_clean":"2024-03-04T17:40:34.016711+0000","last_became_active":"2024-03-04T17:40:34.013287+0000","last_became_peered":"2024-03-04T17:40:34.013287+0000","last_unstale":"2024-03-04T17:40:34.016711+0000","last_undegraded":"2024-03-04T17:40:34.016711+0000","last_fullsized":"2024-03-04T17:40:34.016711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:59:38.174740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-03-04T17:40:34.011011+0000","last_change":"2024-03-04T17:40:34.010119+0000","last_active":"2024-03-04T17:40:34.011011+0000","last_peered":"2024-03-04T17:40:34.011011+0000","last_clean":"2024-03-04T17:40:34.011011+0000","last_became_active":"2024-03-04T17:40:34.009636+0000","last_became_peered":"2024-03-04T17:40:34.009636+0000","last_unstale":"2024-03-04T17:40:34.011011+0000","last_undegraded":"2024-03-04T17:40:34.011011+0000","last_fullsized":"2024-03-04T17:40:34.011011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:32.987918+0000","last_clean_scrub_stamp":"2024-03-04T17:40:32.987918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:11:58.354928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T17:40:35.019441+0000","last_change":"2024-03-04T17:40:26.000245+0000","last_active":"2024-03-04T17:40:35.019441+0000","last_peered":"2024-03-04T17:40:35.019441+0000","last_clean":"2024-03-04T17:40:35.019441+0000","last_became_active":"2024-03-04T17:40:25.998187+0000","last_became_peered":"2024-03-04T17:40:25.998187+0000","last_unstale":"2024-03-04T17:40:35.019441+0000","last_undegraded":"2024-03-04T17:40:35.019441+0000","last_fullsized":"2024-03-04T17:40:35.019441+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T17:40:23.884217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T17:40:23.884217+0000","last_clean_scrub_stamp":"2024-03-04T17:40:23.884217+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:17:30.523252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":59507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":655623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":59507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":59507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":59507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":655623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":59507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":184320,"data_stored":59507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T17:40:38.047 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-04T17:40:38.048 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-04T17:40:38.048 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-04T17:40:38.048 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-04T17:40:38.355 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:40:38.355 INFO:teuthology.orchestra.run.smithi137.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-04T17:40:38.372 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-04T17:40:38.373 INFO:teuthology.run_tasks:Running task openssl_keys... 2024-03-04T17:40:38.384 DEBUG:teuthology.task:Applying overrides for task openssl_keys: {'rgw.client.0': {'ca': 'root', 'client': 'client.0', 'embed-key': True}, 'root': {'client': 'client.0', 'cn': 'teuthology', 'install': ['client.0'], 'key-type': 'rsa:4096'}} 2024-03-04T17:40:38.384 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p /home/ubuntu/cephtest/ca 2024-03-04T17:40:38.389 DEBUG:teuthology.orchestra.run.smithi137:> openssl req -x509 -nodes -newkey rsa:4096 -keyout /home/ubuntu/cephtest/ca/root.key -days 365 -out /home/ubuntu/cephtest/ca/root.crt -subj /CN=teuthology 2024-03-04T17:40:40.085 INFO:teuthology.orchestra.run.smithi137.stderr:............+.........+..+....+...+..+....+......+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.......+...+.....+...+....+...+..+...+.............+..+....+...+............+...+......+...........+....+..+.......+...........+.+...+.................+......+....+.....+...+...+....+...........+...+.+.....+...+.+.....+...................+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...............+............+.....+.+......+......+.....+...+..........+......+........+...+............+................+.....+....+.....+...+....+...+......+..+...+..........+...........+....+.....+.........+.+.........+...+..+.......+...+...+.............................+....+..+....+........+.......+...+......+.....+.+......+.....+...+................+.........+.....+............+....+...+..+......+.........................+..+.+.....+................+...+........+...+....+........+..........+...+..+....+........+..........+......+....................+.+.....+..........+...........+..........+........+.......+...+..............+.....................+......+..................+.............+..................+...............+.........+..+......+.......+...........+....+...+.......................+.......+..+.......+...+..+....+..+.......+......+......+.....+..........+........+.......+...+...+........+.........+.............+.........+...+..+.+...+...............+...............+............+..............+....+......+.....+....+.........+..+.+.....+.........+......+...+.......+..............+.............+...+...+.....+.+...+......+...........+....+.........+........+.........+.+.....+.......+.....+...+.......+..+....+.....+.+...........+.......+...........+.......+..+............+...+..........+..+.............+..+......+.........+.........+.+......+..+.......+........+.......+.........+..+....+...+.....+..........+........+..........+.....+....+........+.+..+...+.+...+........+........................+.+..+.+.........+...+.........+....................+.+.....+....+.......................+......+.+....................+.+........+.+..........................+.......+..+....+...............+.....+....+...+........+.......+.........+........+..................+...+.+...................................+...+......+.........+.+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2024-03-04T17:40:41.085 INFO:teuthology.orchestra.run.smithi137.stderr:.+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+.........+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+..........+............+...+......+.................+...+....+...+..+..............................+......................+........+.+.....+..........+...........+...+.+..+....+.....+..........+...+...+...+..............+.+......+...+........+...+...................+...+...+.....+.+......+.....+............+..........+..+.+..............+...+..........+.....+.+....................+......................+..+.+.........+..+.........+......+......+...+....+......+.........+.....+.......+......+...+..+............+.+...........+....+.........+..+.....................+...+....+............+...+......+..+...+.......+........+.+.........+............+.........+...+..+.+..+......+...................+...............+.....+.......+......+..+..........+.....+......+.+........+......+.+......+..+............+......+.......+......+..+.........+....+.....+.+...+.....+.+............+..+............+.+...+.....+......+......+....+.....+....+.....................+.........+............+..+...+...+....+...+...+..............+..........+..+.............+...+...+..+...+............+....+...+.....+...+....+........+.+......+.....+.+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2024-03-04T17:40:41.101 INFO:teuthology.orchestra.run.smithi137.stderr:----- 2024-03-04T17:40:41.108 DEBUG:teuthology.orchestra.run.smithi137:> sudo cp /home/ubuntu/cephtest/ca/root.crt /usr/local/share/ca-certificates/root.crt 2024-03-04T17:40:41.182 DEBUG:teuthology.orchestra.run.smithi137:> sudo update-ca-certificates 2024-03-04T17:40:41.247 INFO:teuthology.orchestra.run.smithi137.stdout:Updating certificates in /etc/ssl/certs... 2024-03-04T17:40:41.846 INFO:teuthology.orchestra.run.smithi137.stderr:rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL 2024-03-04T17:40:41.866 INFO:teuthology.orchestra.run.smithi137.stdout:1 added, 0 removed; done. 2024-03-04T17:40:41.874 INFO:teuthology.orchestra.run.smithi137.stdout:Running hooks in /etc/ca-certificates/update.d... 2024-03-04T17:40:41.877 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:40:43.423 INFO:teuthology.orchestra.run.smithi137.stdout:Adding debian:root.pem 2024-03-04T17:40:43.439 INFO:teuthology.orchestra.run.smithi137.stdout:done. 2024-03-04T17:40:43.440 INFO:teuthology.orchestra.run.smithi137.stdout:done. 2024-03-04T17:40:43.444 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p /home/ubuntu/cephtest/ca 2024-03-04T17:40:43.492 DEBUG:teuthology.orchestra.run.smithi137:> openssl req -nodes -newkey rsa:2048 -keyout /home/ubuntu/cephtest/ca/rgw.client.0.key -out /home/ubuntu/cephtest/ca/rgw.client.0.csr -subj /CN=smithi137.front.sepia.ceph.com 2024-03-04T17:40:43.587 INFO:teuthology.orchestra.run.smithi137.stderr:.+....+...+...+..+............+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+......................+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+......+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2024-03-04T17:40:43.622 INFO:teuthology.orchestra.run.smithi137.stderr:.........+......+.....+......+....+.........+..+.........+.+...+..+....+...........+....+...........+...+.........+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+...+..........+..+.......+......+...+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*......+...........+....+..+......+...+....+...+.........+.........+..+...+.......+...+..+....+...+...+..+.+......+..............+.......+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2024-03-04T17:40:43.625 INFO:teuthology.orchestra.run.smithi137.stderr:----- 2024-03-04T17:40:43.628 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:40:43.628 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/ca/rgw.client.0.ext 2024-03-04T17:40:43.676 DEBUG:teuthology.orchestra.run.smithi137:> openssl x509 -req -in /home/ubuntu/cephtest/ca/rgw.client.0.csr -CA /home/ubuntu/cephtest/ca/root.crt -CAkey /home/ubuntu/cephtest/ca/root.key -CAcreateserial -out /home/ubuntu/cephtest/ca/rgw.client.0.crt -days 365 -sha256 -extfile /home/ubuntu/cephtest/ca/rgw.client.0.ext 2024-03-04T17:40:43.727 INFO:teuthology.orchestra.run.smithi137.stderr:Certificate request self-signature ok 2024-03-04T17:40:43.727 INFO:teuthology.orchestra.run.smithi137.stderr:subject=CN = smithi137.front.sepia.ceph.com 2024-03-04T17:40:43.739 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/ca/rgw.client.0.csr /home/ubuntu/cephtest/ca/root.srl /home/ubuntu/cephtest/ca/rgw.client.0.ext 2024-03-04T17:40:43.788 DEBUG:teuthology.orchestra.run.smithi137:> openssl verify -CAfile /home/ubuntu/cephtest/ca/root.crt /home/ubuntu/cephtest/ca/rgw.client.0.crt 2024-03-04T17:40:43.840 INFO:teuthology.orchestra.run.smithi137.stdout:/home/ubuntu/cephtest/ca/rgw.client.0.crt: OK 2024-03-04T17:40:43.841 DEBUG:teuthology.orchestra.run.smithi137:> cat /home/ubuntu/cephtest/ca/rgw.client.0.key >> /home/ubuntu/cephtest/ca/rgw.client.0.crt 2024-03-04T17:40:43.888 INFO:teuthology.run_tasks:Running task rgw... 2024-03-04T17:40:43.901 DEBUG:tasks.rgw:config is {'client.0': {'ssl certificate': 'rgw.client.0', 'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2024-03-04T17:40:43.901 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-03-04T17:40:43.902 INFO:tasks.rgw:Creating data pools 2024-03-04T17:40:43.902 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-03-04T17:40:43.902 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster ceph 2024-03-04T17:40:45.046 INFO:teuthology.orchestra.run.smithi137.stderr:pool 'default.rgw.buckets.data' created 2024-03-04T17:40:45.081 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-03-04T17:40:46.057 INFO:teuthology.orchestra.run.smithi137.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-03-04T17:40:46.097 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-03-04T17:40:47.068 INFO:teuthology.orchestra.run.smithi137.stderr:pool 'default.rgw.buckets.index' created 2024-03-04T17:40:47.103 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-03-04T17:40:48.091 INFO:teuthology.orchestra.run.smithi137.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-03-04T17:40:48.132 DEBUG:tasks.rgw:Pools created 2024-03-04T17:40:48.132 INFO:tasks.rgw:Configuring compression type = random 2024-03-04T17:40:48.132 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-03-04T17:40:48.132 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-03-04T17:40:48.132 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-03-04T17:40:48.190 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:40:48.191 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:40:50.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.175+0000 7fb6e5e21640 0 -- 172.21.15.137:0/1141909319 >> [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] conn(0x7fb5b8065850 msgr2=0x7fb5b8067d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:40:50.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.179+0000 7fb6e7b08980 20 rados->read ofs=0 len=0 2024-03-04T17:40:50.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.179+0000 7fb6e5620640 0 -- 172.21.15.137:0/3726051771 >> [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] conn(0x7fb5ac065940 msgr2=0x7fb5ac067e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:40:50.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.183+0000 7fb6e7b08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:40:50.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.183+0000 7fb6e7b08980 20 realm 2024-03-04T17:40:50.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.183+0000 7fb6e7b08980 20 rados->read ofs=0 len=0 2024-03-04T17:40:50.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.183+0000 7fb6e7b08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:40:50.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.183+0000 7fb6e7b08980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:40:50.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.183+0000 7fb6e7b08980 20 rados->read ofs=0 len=0 2024-03-04T17:40:50.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.183+0000 7fb6e7b08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:40:50.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.183+0000 7fb6e7b08980 20 rados->read ofs=0 len=0 2024-03-04T17:40:50.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.183+0000 7fb6e7b08980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:40:50.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.183+0000 7fb6e7b08980 20 rados->read ofs=0 len=0 2024-03-04T17:40:50.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.183+0000 7fb6e7b08980 20 rados_obj.operate() r=0 bl.length=875 2024-03-04T17:40:50.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.183+0000 7fb6e7b08980 20 searching for the correct realm 2024-03-04T17:40:50.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.203+0000 7fb6e7b08980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:40:50.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.203+0000 7fb6e7b08980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:40:50.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.203+0000 7fb6e7b08980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:40:50.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.203+0000 7fb6e7b08980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:40:50.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.203+0000 7fb6e7b08980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:40:50.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.203+0000 7fb6e7b08980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:40:50.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.203+0000 7fb6e7b08980 20 rados->read ofs=0 len=0 2024-03-04T17:40:50.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.203+0000 7fb6e7b08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:40:50.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.203+0000 7fb6e7b08980 20 rados->read ofs=0 len=0 2024-03-04T17:40:50.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.203+0000 7fb6e7b08980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:40:50.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.203+0000 7fb6e7b08980 20 rados->read ofs=0 len=0 2024-03-04T17:40:50.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.207+0000 7fb6e7b08980 20 rados_obj.operate() r=0 bl.length=398 2024-03-04T17:40:50.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.207+0000 7fb6e7b08980 20 zone default found 2024-03-04T17:40:50.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.207+0000 7fb6e7b08980 4 Realm: () 2024-03-04T17:40:50.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.207+0000 7fb6e7b08980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:40:50.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.207+0000 7fb6e7b08980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:40:50.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.207+0000 7fb6e7b08980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:40:50.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.207+0000 7fb6e7b08980 20 zonegroup default 2024-03-04T17:40:50.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.207+0000 7fb6e7b08980 20 rados->read ofs=0 len=0 2024-03-04T17:40:50.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.207+0000 7fb6e7b08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:40:50.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:50.207+0000 7fb6e7b08980 20 rados->read ofs=0 len=0 2024-03-04T17:40:52.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:52.123+0000 7fb6e7b08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:40:52.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:52.123+0000 7fb6e7b08980 20 rados->read ofs=0 len=0 2024-03-04T17:40:52.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:52.123+0000 7fb6e7b08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:40:52.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:52.123+0000 7fb6e7b08980 20 started sync module instance, tier type = 2024-03-04T17:40:52.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:52.123+0000 7fb6e7b08980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:40:54.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.191+0000 7fb6e7b08980 20 add_watcher() i=0 2024-03-04T17:40:54.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.191+0000 7fb6e7b08980 20 add_watcher() i=1 2024-03-04T17:40:54.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.191+0000 7fb6e7b08980 20 add_watcher() i=2 2024-03-04T17:40:54.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.191+0000 7fb6e7b08980 20 add_watcher() i=3 2024-03-04T17:40:54.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.191+0000 7fb6e7b08980 20 add_watcher() i=4 2024-03-04T17:40:54.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.191+0000 7fb6e7b08980 20 add_watcher() i=5 2024-03-04T17:40:54.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.191+0000 7fb6e7b08980 20 add_watcher() i=6 2024-03-04T17:40:54.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.191+0000 7fb6e7b08980 20 add_watcher() i=7 2024-03-04T17:40:54.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.191+0000 7fb6e7b08980 2 all 8 watchers are set, enabling cache 2024-03-04T17:40:54.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.191+0000 7fb6e7b08980 5 tl::expected, obj_version>, boost::system::error_code> logback_generations::read(const DoutPrefixProvider*, optional_yield):410: oid=data_loggenerations_metadata not found 2024-03-04T17:40:54.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.191+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.0 2024-03-04T17:40:54.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.191+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.0 does not exist 2024-03-04T17:40:54.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.1 2024-03-04T17:40:54.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.1 does not exist 2024-03-04T17:40:54.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.2 2024-03-04T17:40:54.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.2 does not exist 2024-03-04T17:40:54.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.3 2024-03-04T17:40:54.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.3 does not exist 2024-03-04T17:40:54.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.4 2024-03-04T17:40:54.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.4 does not exist 2024-03-04T17:40:54.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.5 2024-03-04T17:40:54.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.5 does not exist 2024-03-04T17:40:54.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.6 2024-03-04T17:40:54.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.195+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.6 does not exist 2024-03-04T17:40:54.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.7 2024-03-04T17:40:54.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.7 does not exist 2024-03-04T17:40:54.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.8 2024-03-04T17:40:54.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.8 does not exist 2024-03-04T17:40:54.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.9 2024-03-04T17:40:54.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.9 does not exist 2024-03-04T17:40:54.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.10 2024-03-04T17:40:54.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.10 does not exist 2024-03-04T17:40:54.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.11 2024-03-04T17:40:54.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.11 does not exist 2024-03-04T17:40:54.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.12 2024-03-04T17:40:54.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.199+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.12 does not exist 2024-03-04T17:40:54.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.13 2024-03-04T17:40:54.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.13 does not exist 2024-03-04T17:40:54.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.14 2024-03-04T17:40:54.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.14 does not exist 2024-03-04T17:40:54.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.15 2024-03-04T17:40:54.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.15 does not exist 2024-03-04T17:40:54.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.16 2024-03-04T17:40:54.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.16 does not exist 2024-03-04T17:40:54.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.17 2024-03-04T17:40:54.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.17 does not exist 2024-03-04T17:40:54.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.18 2024-03-04T17:40:54.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.18 does not exist 2024-03-04T17:40:54.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.19 2024-03-04T17:40:54.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.19 does not exist 2024-03-04T17:40:54.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.20 2024-03-04T17:40:54.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.203+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.20 does not exist 2024-03-04T17:40:54.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.21 2024-03-04T17:40:54.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.21 does not exist 2024-03-04T17:40:54.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.22 2024-03-04T17:40:54.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.22 does not exist 2024-03-04T17:40:54.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.23 2024-03-04T17:40:54.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.23 does not exist 2024-03-04T17:40:54.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.24 2024-03-04T17:40:54.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.24 does not exist 2024-03-04T17:40:54.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.25 2024-03-04T17:40:54.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.25 does not exist 2024-03-04T17:40:54.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.26 2024-03-04T17:40:54.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.207+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.26 does not exist 2024-03-04T17:40:54.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.27 2024-03-04T17:40:54.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.27 does not exist 2024-03-04T17:40:54.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.28 2024-03-04T17:40:54.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.28 does not exist 2024-03-04T17:40:54.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.29 2024-03-04T17:40:54.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.29 does not exist 2024-03-04T17:40:54.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.30 2024-03-04T17:40:54.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.30 does not exist 2024-03-04T17:40:54.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.31 2024-03-04T17:40:54.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.31 does not exist 2024-03-04T17:40:54.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.32 2024-03-04T17:40:54.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.32 does not exist 2024-03-04T17:40:54.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.33 2024-03-04T17:40:54.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.211+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.33 does not exist 2024-03-04T17:40:54.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.34 2024-03-04T17:40:54.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.34 does not exist 2024-03-04T17:40:54.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.35 2024-03-04T17:40:54.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.35 does not exist 2024-03-04T17:40:54.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.36 2024-03-04T17:40:54.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.36 does not exist 2024-03-04T17:40:54.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.37 2024-03-04T17:40:54.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.37 does not exist 2024-03-04T17:40:54.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.38 2024-03-04T17:40:54.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.38 does not exist 2024-03-04T17:40:54.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.39 2024-03-04T17:40:54.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.39 does not exist 2024-03-04T17:40:54.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.40 2024-03-04T17:40:54.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.215+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.40 does not exist 2024-03-04T17:40:54.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.41 2024-03-04T17:40:54.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.41 does not exist 2024-03-04T17:40:54.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.42 2024-03-04T17:40:54.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.42 does not exist 2024-03-04T17:40:54.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.43 2024-03-04T17:40:54.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.43 does not exist 2024-03-04T17:40:54.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.44 2024-03-04T17:40:54.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.44 does not exist 2024-03-04T17:40:54.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.45 2024-03-04T17:40:54.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.45 does not exist 2024-03-04T17:40:54.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.46 2024-03-04T17:40:54.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.46 does not exist 2024-03-04T17:40:54.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.47 2024-03-04T17:40:54.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.219+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.47 does not exist 2024-03-04T17:40:54.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.48 2024-03-04T17:40:54.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.48 does not exist 2024-03-04T17:40:54.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.49 2024-03-04T17:40:54.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.49 does not exist 2024-03-04T17:40:54.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.50 2024-03-04T17:40:54.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.50 does not exist 2024-03-04T17:40:54.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.51 2024-03-04T17:40:54.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.51 does not exist 2024-03-04T17:40:54.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.52 2024-03-04T17:40:54.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.52 does not exist 2024-03-04T17:40:54.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.53 2024-03-04T17:40:54.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.223+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.53 does not exist 2024-03-04T17:40:54.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.54 2024-03-04T17:40:54.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.54 does not exist 2024-03-04T17:40:54.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.55 2024-03-04T17:40:54.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.55 does not exist 2024-03-04T17:40:54.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.56 2024-03-04T17:40:54.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.56 does not exist 2024-03-04T17:40:54.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.57 2024-03-04T17:40:54.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.57 does not exist 2024-03-04T17:40:54.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.58 2024-03-04T17:40:54.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.58 does not exist 2024-03-04T17:40:54.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.59 2024-03-04T17:40:54.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.59 does not exist 2024-03-04T17:40:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.60 2024-03-04T17:40:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.227+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.60 does not exist 2024-03-04T17:40:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.61 2024-03-04T17:40:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.61 does not exist 2024-03-04T17:40:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.62 2024-03-04T17:40:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.62 does not exist 2024-03-04T17:40:54.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.63 2024-03-04T17:40:54.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.63 does not exist 2024-03-04T17:40:54.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.64 2024-03-04T17:40:54.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.64 does not exist 2024-03-04T17:40:54.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.65 2024-03-04T17:40:54.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.65 does not exist 2024-03-04T17:40:54.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.66 2024-03-04T17:40:54.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.231+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.66 does not exist 2024-03-04T17:40:54.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.67 2024-03-04T17:40:54.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.67 does not exist 2024-03-04T17:40:54.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.68 2024-03-04T17:40:54.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.68 does not exist 2024-03-04T17:40:54.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.69 2024-03-04T17:40:54.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.69 does not exist 2024-03-04T17:40:54.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.70 2024-03-04T17:40:54.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.70 does not exist 2024-03-04T17:40:54.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.71 2024-03-04T17:40:54.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.235+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.71 does not exist 2024-03-04T17:40:54.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.72 2024-03-04T17:40:54.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.72 does not exist 2024-03-04T17:40:54.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.73 2024-03-04T17:40:54.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.73 does not exist 2024-03-04T17:40:54.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.74 2024-03-04T17:40:54.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.74 does not exist 2024-03-04T17:40:54.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.75 2024-03-04T17:40:54.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.75 does not exist 2024-03-04T17:40:54.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.76 2024-03-04T17:40:54.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.76 does not exist 2024-03-04T17:40:54.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.77 2024-03-04T17:40:54.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.77 does not exist 2024-03-04T17:40:54.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.78 2024-03-04T17:40:54.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.239+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.78 does not exist 2024-03-04T17:40:54.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.79 2024-03-04T17:40:54.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.79 does not exist 2024-03-04T17:40:54.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.80 2024-03-04T17:40:54.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.80 does not exist 2024-03-04T17:40:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.81 2024-03-04T17:40:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.81 does not exist 2024-03-04T17:40:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.82 2024-03-04T17:40:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.82 does not exist 2024-03-04T17:40:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.83 2024-03-04T17:40:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.83 does not exist 2024-03-04T17:40:54.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.84 2024-03-04T17:40:54.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.243+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.84 does not exist 2024-03-04T17:40:54.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.85 2024-03-04T17:40:54.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.85 does not exist 2024-03-04T17:40:54.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.86 2024-03-04T17:40:54.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.86 does not exist 2024-03-04T17:40:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.87 2024-03-04T17:40:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.87 does not exist 2024-03-04T17:40:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.88 2024-03-04T17:40:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.88 does not exist 2024-03-04T17:40:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.89 2024-03-04T17:40:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.89 does not exist 2024-03-04T17:40:54.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.90 2024-03-04T17:40:54.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.247+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.90 does not exist 2024-03-04T17:40:54.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.91 2024-03-04T17:40:54.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.91 does not exist 2024-03-04T17:40:54.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.92 2024-03-04T17:40:54.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.92 does not exist 2024-03-04T17:40:54.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.93 2024-03-04T17:40:54.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.93 does not exist 2024-03-04T17:40:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.94 2024-03-04T17:40:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.94 does not exist 2024-03-04T17:40:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.95 2024-03-04T17:40:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.251+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.95 does not exist 2024-03-04T17:40:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.96 2024-03-04T17:40:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.96 does not exist 2024-03-04T17:40:54.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.97 2024-03-04T17:40:54.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.97 does not exist 2024-03-04T17:40:54.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.98 2024-03-04T17:40:54.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.98 does not exist 2024-03-04T17:40:54.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.99 2024-03-04T17:40:54.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.99 does not exist 2024-03-04T17:40:54.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.100 2024-03-04T17:40:54.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.100 does not exist 2024-03-04T17:40:54.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.101 2024-03-04T17:40:54.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.255+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.101 does not exist 2024-03-04T17:40:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.102 2024-03-04T17:40:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.102 does not exist 2024-03-04T17:40:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.103 2024-03-04T17:40:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.103 does not exist 2024-03-04T17:40:54.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.104 2024-03-04T17:40:54.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.104 does not exist 2024-03-04T17:40:54.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.105 2024-03-04T17:40:54.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.105 does not exist 2024-03-04T17:40:54.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.106 2024-03-04T17:40:54.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.259+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.106 does not exist 2024-03-04T17:40:54.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.107 2024-03-04T17:40:54.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.107 does not exist 2024-03-04T17:40:54.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.108 2024-03-04T17:40:54.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.108 does not exist 2024-03-04T17:40:54.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.109 2024-03-04T17:40:54.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.109 does not exist 2024-03-04T17:40:54.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.110 2024-03-04T17:40:54.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.110 does not exist 2024-03-04T17:40:54.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.111 2024-03-04T17:40:54.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.263+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.111 does not exist 2024-03-04T17:40:54.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.112 2024-03-04T17:40:54.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.112 does not exist 2024-03-04T17:40:54.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.113 2024-03-04T17:40:54.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.113 does not exist 2024-03-04T17:40:54.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.114 2024-03-04T17:40:54.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.114 does not exist 2024-03-04T17:40:54.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.115 2024-03-04T17:40:54.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.115 does not exist 2024-03-04T17:40:54.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.116 2024-03-04T17:40:54.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.267+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.116 does not exist 2024-03-04T17:40:54.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.117 2024-03-04T17:40:54.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.117 does not exist 2024-03-04T17:40:54.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.118 2024-03-04T17:40:54.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.118 does not exist 2024-03-04T17:40:54.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.119 2024-03-04T17:40:54.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.119 does not exist 2024-03-04T17:40:54.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.120 2024-03-04T17:40:54.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.120 does not exist 2024-03-04T17:40:54.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.121 2024-03-04T17:40:54.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.271+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.121 does not exist 2024-03-04T17:40:54.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.122 2024-03-04T17:40:54.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.122 does not exist 2024-03-04T17:40:54.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.123 2024-03-04T17:40:54.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.123 does not exist 2024-03-04T17:40:54.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.124 2024-03-04T17:40:54.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.124 does not exist 2024-03-04T17:40:54.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.125 2024-03-04T17:40:54.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.125 does not exist 2024-03-04T17:40:54.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.126 2024-03-04T17:40:54.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.126 does not exist 2024-03-04T17:40:54.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.127 2024-03-04T17:40:54.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.275+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.279+0000 7fb6e7b08980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.127 does not exist 2024-03-04T17:40:54.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.279+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::create(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, std::optional >, bool, uint64_t, uint64_t):1211 entering 2024-03-04T17:40:54.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.279+0000 7fb6e7b08980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-03-04T17:40:54.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.291+0000 7fb5977e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:40:54.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.291+0000 7fb6e7b08980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:40:54.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:54.291+0000 7fb6e7b08980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:40:57.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.143+0000 7fb6e7b08980 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2024-03-04T17:40:57.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.143+0000 7fb6e7b08980 5 note: GC not initialized 2024-03-04T17:40:57.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.159+0000 7fb5967e4640 20 reqs_thread_entry: start 2024-03-04T17:40:57.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.159+0000 7fb6e7b08980 20 init_complete bucket index max shards: 1 2024-03-04T17:40:57.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.159+0000 7fb6e7b08980 20 Filter name: none 2024-03-04T17:40:57.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.159+0000 7fb4f7fff640 20 reqs_thread_entry: start 2024-03-04T17:40:57.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.211+0000 7fb5977e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:40:57.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.215+0000 7fb6e7b08980 20 remove_watcher() i=0 2024-03-04T17:40:57.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.215+0000 7fb6e7b08980 2 removed watcher, disabling cache 2024-03-04T17:40:57.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.219+0000 7fb6e7b08980 20 remove_watcher() i=1 2024-03-04T17:40:57.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.219+0000 7fb6e7b08980 20 remove_watcher() i=2 2024-03-04T17:40:57.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.223+0000 7fb6e7b08980 20 remove_watcher() i=3 2024-03-04T17:40:57.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.223+0000 7fb6e7b08980 20 remove_watcher() i=4 2024-03-04T17:40:57.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.227+0000 7fb6e7b08980 20 remove_watcher() i=5 2024-03-04T17:40:57.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.231+0000 7fb6e7b08980 20 remove_watcher() i=6 2024-03-04T17:40:57.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.235+0000 7fb6e7b08980 20 remove_watcher() i=7 2024-03-04T17:40:57.252 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-04T17:40:57.253 DEBUG:tasks.util.rgw: json result: [] 2024-03-04T17:40:57.253 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-03-04T17:40:57.253 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-03-04T17:40:57.253 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2024-03-04T17:40:57.293 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:40:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:40:57.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.323+0000 7f227b706980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:40:57.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.323+0000 7f227b706980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:40:57.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.327+0000 7f21377fe640 20 reqs_thread_entry: start 2024-03-04T17:40:57.348 INFO:teuthology.orchestra.run.smithi137.stdout:{"id":"f85b0d2e-c665-46d3-ae77-23c29725ed4a","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"","notif_pool":"default.rgw.log:notif"} 2024-03-04T17:40:57.348 DEBUG:tasks.util.rgw: json result: {'id': 'f85b0d2e-c665-46d3-ae77-23c29725ed4a', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-03-04T17:40:57.348 INFO:tasks.rgw:Preparing directory for rgw datacache at /tmp/rgw_datacache 2024-03-04T17:40:57.348 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p /tmp/rgw_datacache 2024-03-04T17:40:57.396 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a+rwx /tmp/rgw_datacache 2024-03-04T17:40:57.449 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-03-04T17:40:57.449 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-03-04T17:40:57.449 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-03-04T17:40:57.525 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:40:57.525 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:40:57.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.563+0000 7fedd5c93980 20 rados->read ofs=0 len=0 2024-03-04T17:40:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.563+0000 7fedd5c93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:40:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.563+0000 7fedd5c93980 20 realm 2024-03-04T17:40:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.563+0000 7fedd5c93980 20 rados->read ofs=0 len=0 2024-03-04T17:40:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.563+0000 7fedd5c93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:40:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.563+0000 7fedd5c93980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:40:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.563+0000 7fedd5c93980 20 rados->read ofs=0 len=0 2024-03-04T17:40:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.567+0000 7fedd5c93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:40:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.567+0000 7fedd5c93980 20 rados->read ofs=0 len=0 2024-03-04T17:40:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.567+0000 7fedd5c93980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:40:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.567+0000 7fedd5c93980 20 rados->read ofs=0 len=0 2024-03-04T17:40:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.567+0000 7fedd5c93980 20 rados_obj.operate() r=0 bl.length=891 2024-03-04T17:40:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.567+0000 7fedd5c93980 20 searching for the correct realm 2024-03-04T17:40:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.571+0000 7fedd3fac640 0 -- 172.21.15.137:0/333261967 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55f66c1fe680 msgr2=0x55f66c206080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:40:57.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.579+0000 7fedd2faa640 0 -- 172.21.15.137:0/333261967 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x55f66c1ff040 msgr2=0x55f66c201450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:40:57.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:40:57.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:40:57.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:40:57.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:40:57.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:40:57.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:40:57.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 rados->read ofs=0 len=0 2024-03-04T17:40:57.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:40:57.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 rados->read ofs=0 len=0 2024-03-04T17:40:57.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:40:57.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 rados->read ofs=0 len=0 2024-03-04T17:40:57.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 rados_obj.operate() r=0 bl.length=398 2024-03-04T17:40:57.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 zone default found 2024-03-04T17:40:57.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 4 Realm: () 2024-03-04T17:40:57.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:40:57.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:40:57.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:40:57.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 zonegroup default 2024-03-04T17:40:57.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.587+0000 7fedd5c93980 20 rados->read ofs=0 len=0 2024-03-04T17:40:57.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.591+0000 7fedd5c93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:40:57.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.591+0000 7fedd5c93980 20 rados->read ofs=0 len=0 2024-03-04T17:40:57.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.591+0000 7fedd5c93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:40:57.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.591+0000 7fedd5c93980 20 rados->read ofs=0 len=0 2024-03-04T17:40:57.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.591+0000 7fedd5c93980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:40:57.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.591+0000 7fedd5c93980 20 started sync module instance, tier type = 2024-03-04T17:40:57.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.591+0000 7fedd5c93980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:40:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.611+0000 7fedd5c93980 20 add_watcher() i=0 2024-03-04T17:40:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.611+0000 7fedd5c93980 20 add_watcher() i=1 2024-03-04T17:40:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.611+0000 7fedd5c93980 20 add_watcher() i=2 2024-03-04T17:40:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.611+0000 7fedd5c93980 20 add_watcher() i=3 2024-03-04T17:40:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.611+0000 7fedd5c93980 20 add_watcher() i=4 2024-03-04T17:40:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.611+0000 7fedd5c93980 20 add_watcher() i=5 2024-03-04T17:40:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.611+0000 7fedd5c93980 20 add_watcher() i=6 2024-03-04T17:40:57.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.615+0000 7fedd5c93980 20 add_watcher() i=7 2024-03-04T17:40:57.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.615+0000 7fedd5c93980 2 all 8 watchers are set, enabling cache 2024-03-04T17:40:57.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.619+0000 7fec89fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:40:57.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.619+0000 7fedd5c93980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:40:57.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.619+0000 7fedd5c93980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:40:57.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.619+0000 7fedd5c93980 5 note: GC not initialized 2024-03-04T17:40:57.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.619+0000 7fec88fe1640 20 reqs_thread_entry: start 2024-03-04T17:40:57.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.619+0000 7fedd5c93980 20 init_complete bucket index max shards: 1 2024-03-04T17:40:57.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.619+0000 7fedd5c93980 20 Filter name: none 2024-03-04T17:40:57.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.619+0000 7fec027fc640 20 reqs_thread_entry: start 2024-03-04T17:40:57.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.639+0000 7fec89fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:40:57.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.643+0000 7fedd5c93980 20 remove_watcher() i=0 2024-03-04T17:40:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.643+0000 7fedd5c93980 2 removed watcher, disabling cache 2024-03-04T17:40:57.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.647+0000 7fedd5c93980 20 remove_watcher() i=1 2024-03-04T17:40:57.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.647+0000 7fedd5c93980 20 remove_watcher() i=2 2024-03-04T17:40:57.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.651+0000 7fedd5c93980 20 remove_watcher() i=3 2024-03-04T17:40:57.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.651+0000 7fedd5c93980 20 remove_watcher() i=4 2024-03-04T17:40:57.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.655+0000 7fedd5c93980 20 remove_watcher() i=5 2024-03-04T17:40:57.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.659+0000 7fedd5c93980 20 remove_watcher() i=6 2024-03-04T17:40:57.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.659+0000 7fedd5c93980 20 remove_watcher() i=7 2024-03-04T17:40:57.677 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-04T17:40:57.678 DEBUG:tasks.util.rgw: json result: [] 2024-03-04T17:40:57.678 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2024-03-04T17:40:57.678 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2024-03-04T17:40:57.678 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2024-03-04T17:40:57.678 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2024-03-04T17:40:57.759 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:40:57.759 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:40:57.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.787+0000 7f218fcb2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:40:57.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.787+0000 7f218fcb2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:40:57.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.791+0000 7f204b7fe640 20 reqs_thread_entry: start 2024-03-04T17:40:57.810 INFO:teuthology.orchestra.run.smithi137.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2024-03-04T17:40:57.810 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['LUKEWARM', 'STANDARD']}}] 2024-03-04T17:40:57.810 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2024-03-04T17:40:57.810 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2024-03-04T17:40:57.810 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2024-03-04T17:40:57.894 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:40:57.894 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:40:57.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.919+0000 7f67fa799980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:40:57.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.919+0000 7f67fa799980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:40:57.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:57.923+0000 7f665e7fc640 20 reqs_thread_entry: start 2024-03-04T17:40:57.944 INFO:teuthology.orchestra.run.smithi137.stdout:{"id":"f85b0d2e-c665-46d3-ae77-23c29725ed4a","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"","notif_pool":"default.rgw.log:notif"} 2024-03-04T17:40:57.944 DEBUG:tasks.util.rgw: json result: {'id': 'f85b0d2e-c665-46d3-ae77-23c29725ed4a', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-03-04T17:40:57.944 INFO:tasks.rgw:Configuring storage class type = FROZEN 2024-03-04T17:40:57.944 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2024-03-04T17:40:57.944 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2024-03-04T17:40:57.944 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2024-03-04T17:40:58.020 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:40:58.021 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:40:58.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:58.047+0000 7ff31e614980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:40:58.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:58.047+0000 7ff31e614980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:40:58.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:58.051+0000 7ff1627fc640 20 reqs_thread_entry: start 2024-03-04T17:40:58.068 INFO:teuthology.orchestra.run.smithi137.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2024-03-04T17:40:58.068 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['FROZEN', 'LUKEWARM', 'STANDARD']}}] 2024-03-04T17:40:58.068 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2024-03-04T17:40:58.068 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2024-03-04T17:40:58.069 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2024-03-04T17:40:58.147 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:40:58.147 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:40:58.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:58.171+0000 7f144b772980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:40:58.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:58.171+0000 7f144b772980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:40:58.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:40:58.175+0000 7f127ffff640 20 reqs_thread_entry: start 2024-03-04T17:40:58.197 INFO:teuthology.orchestra.run.smithi137.stdout:{"id":"f85b0d2e-c665-46d3-ae77-23c29725ed4a","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"","notif_pool":"default.rgw.log:notif"} 2024-03-04T17:40:58.197 DEBUG:tasks.util.rgw: json result: {'id': 'f85b0d2e-c665-46d3-ae77-23c29725ed4a', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'FROZEN': {'data_pool': 'default.rgw.buckets.data.frozen'}, 'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-03-04T17:40:58.197 INFO:tasks.rgw:Starting rgw... 2024-03-04T17:40:58.197 INFO:tasks.rgw:rgw client.0 config is {'ssl certificate': 'rgw.client.0', 'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2024-03-04T17:40:58.197 INFO:tasks.rgw:Using beast as radosgw frontend 2024-03-04T17:40:58.197 DEBUG:teuthology.orchestra.run.smithi137:> sudo echo -n https://smithi137.front.sepia.ceph.com:443 | sudo tee /home/ubuntu/cephtest/url_file 2024-03-04T17:40:58.251 INFO:teuthology.orchestra.run.smithi137.stdout:https://smithi137.front.sepia.ceph.com:443 2024-03-04T17:40:58.251 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-03-04T17:40:58.304 DEBUG:teuthology.orchestra.run.smithi137:> sudo keytool -import -alias smithi137.front.sepia.ceph.com -keystore $(readlink -e $(dirname $(readlink -e $(which keytool)))/../lib/security/cacerts) -file /home/ubuntu/cephtest/ca/rgw.client.0.crt -storepass changeit 2024-03-04T17:40:58.605 INFO:teuthology.orchestra.run.smithi137.stderr:Warning: use -cacerts option to access cacerts keystore 2024-03-04T17:40:58.943 INFO:teuthology.orchestra.run.smithi137.stderr:Certificate was added to keystore 2024-03-04T17:40:58.961 DEBUG:tasks.ceph_manager:running ceph.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/ceph.client.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2024-03-04T17:40:58.961 INFO:tasks.rgw.client.0:Restarting daemon 2024-03-04T17:40:58.962 DEBUG:teuthology.orchestra.run.smithi137:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast ssl_certificate=/home/ubuntu/cephtest/ca/rgw.client.0.crt ssl_port=443' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-03-04T17:40:58.964 INFO:tasks.rgw.client.0:Started 2024-03-04T17:40:58.964 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on https://smithi137.front.sepia.ceph.com:443/ 2024-03-04T17:40:58.964 DEBUG:teuthology.orchestra.run.smithi137:> curl https://smithi137.front.sepia.ceph.com:443/ 2024-03-04T17:40:58.987 INFO:teuthology.orchestra.run.smithi137.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-04T17:40:58.987 INFO:teuthology.orchestra.run.smithi137.stderr: Dload Upload Total Spent Left Speed 2024-03-04T17:40:58.987 INFO:teuthology.orchestra.run.smithi137.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-03-04T17:40:58.988 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-03-04T17:40:58.988 INFO:teuthology.orchestra.run.smithi137.stderr:curl: (7) Failed to connect to smithi137.front.sepia.ceph.com port 443 after 0 ms: Connection refused 2024-03-04T17:40:59.989 DEBUG:teuthology.orchestra.run.smithi137:> curl https://smithi137.front.sepia.ceph.com:443/ 2024-03-04T17:41:00.000 INFO:teuthology.orchestra.run.smithi137.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-04T17:41:00.000 INFO:teuthology.orchestra.run.smithi137.stderr: Dload Upload Total Spent Left Speed 2024-03-04T17:41:00.000 INFO:teuthology.orchestra.run.smithi137.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-03-04T17:41:00.000 INFO:teuthology.orchestra.run.smithi137.stderr:curl: (7) Failed to connect to smithi137.front.sepia.ceph.com port 443 after 0 ms: Connection refused 2024-03-04T17:41:00.001 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-03-04T17:41:02.002 DEBUG:teuthology.orchestra.run.smithi137:> curl https://smithi137.front.sepia.ceph.com:443/ 2024-03-04T17:41:02.013 INFO:teuthology.orchestra.run.smithi137.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-04T17:41:02.013 INFO:teuthology.orchestra.run.smithi137.stderr: Dload Upload Total Spent Left Speed 2024-03-04T17:41:02.013 INFO:teuthology.orchestra.run.smithi137.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-03-04T17:41:02.013 INFO:teuthology.orchestra.run.smithi137.stderr:curl: (7) Failed to connect to smithi137.front.sepia.ceph.com port 443 after 0 ms: Connection refused 2024-03-04T17:41:02.014 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-03-04T17:41:06.015 DEBUG:teuthology.orchestra.run.smithi137:> curl https://smithi137.front.sepia.ceph.com:443/ 2024-03-04T17:41:06.025 INFO:teuthology.orchestra.run.smithi137.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-04T17:41:06.025 INFO:teuthology.orchestra.run.smithi137.stderr: Dload Upload Total Spent Left Speed 2024-03-04T17:41:06.026 INFO:teuthology.orchestra.run.smithi137.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-03-04T17:41:06.026 INFO:teuthology.orchestra.run.smithi137.stderr:curl: (7) Failed to connect to smithi137.front.sepia.ceph.com port 443 after 0 ms: Connection refused 2024-03-04T17:41:06.026 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-03-04T17:41:14.027 DEBUG:teuthology.orchestra.run.smithi137:> curl https://smithi137.front.sepia.ceph.com:443/ 2024-03-04T17:41:14.037 INFO:teuthology.orchestra.run.smithi137.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-04T17:41:14.037 INFO:teuthology.orchestra.run.smithi137.stderr: Dload Upload Total Spent Left Speed 2024-03-04T17:41:14.037 INFO:teuthology.orchestra.run.smithi137.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-03-04T17:41:14.037 INFO:teuthology.orchestra.run.smithi137.stderr:curl: (7) Failed to connect to smithi137.front.sepia.ceph.com port 443 after 0 ms: Connection refused 2024-03-04T17:41:14.037 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-03-04T17:41:23.611 INFO:tasks.rgw.client.0.smithi137.stdout:2024-03-04T17:41:23.247+0000 8e90e80 -1 ssl_private_key was not found: rgw/cert//default.key 2024-03-04T17:41:30.039 DEBUG:teuthology.orchestra.run.smithi137:> curl https://smithi137.front.sepia.ceph.com:443/ 2024-03-04T17:41:30.050 INFO:teuthology.orchestra.run.smithi137.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-04T17:41:30.050 INFO:teuthology.orchestra.run.smithi137.stderr: Dload Upload Total Spent Left Speed 2024-03-04T17:41:31.196 INFO:teuthology.orchestra.run.smithi137.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 100 214 0 214 0 0 186 0 --:--:-- 0:00:01 --:--:-- 186 2024-03-04T17:41:31.200 INFO:teuthology.orchestra.run.smithi137.stdout:anonymous 2024-03-04T17:41:31.201 INFO:teuthology.run_tasks:Running task tox... 2024-03-04T17:41:31.211 INFO:tasks.tox:Deploying tox from pip... 2024-03-04T17:41:31.211 DEBUG:teuthology.orchestra.run.smithi137:> python3 -m venv /home/ubuntu/cephtest/tox-venv 2024-03-04T17:41:34.085 DEBUG:teuthology.orchestra.run.smithi137:> source /home/ubuntu/cephtest/tox-venv/bin/activate && pip install tox==3.15.0 2024-03-04T17:41:34.815 INFO:teuthology.orchestra.run.smithi137.stdout:Collecting tox==3.15.0 2024-03-04T17:41:34.932 INFO:teuthology.orchestra.run.smithi137.stdout: Downloading tox-3.15.0-py2.py3-none-any.whl (82 kB) 2024-03-04T17:41:35.025 INFO:teuthology.orchestra.run.smithi137.stdout: --------------------------------------- 82.1/82.1 KB 868.2 kB/s eta 0:00:00 2024-03-04T17:41:35.194 INFO:teuthology.orchestra.run.smithi137.stdout:Collecting pluggy<1,>=0.12.0 2024-03-04T17:41:35.248 INFO:teuthology.orchestra.run.smithi137.stdout: Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB) 2024-03-04T17:41:35.574 INFO:teuthology.orchestra.run.smithi137.stdout:Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 2024-03-04T17:41:35.623 INFO:teuthology.orchestra.run.smithi137.stdout: Downloading virtualenv-20.25.1-py3-none-any.whl (3.8 MB) 2024-03-04T17:41:36.578 INFO:teuthology.orchestra.run.smithi137.stdout: ---------------------------------------- 3.8/3.8 MB 4.0 MB/s eta 0:00:00 2024-03-04T17:41:36.762 INFO:teuthology.orchestra.run.smithi137.stdout:Collecting py<2,>=1.4.17 2024-03-04T17:41:36.817 INFO:teuthology.orchestra.run.smithi137.stdout: Downloading py-1.11.0-py2.py3-none-any.whl (98 kB) 2024-03-04T17:41:36.860 INFO:teuthology.orchestra.run.smithi137.stdout: ---------------------------------------- 98.7/98.7 KB 2.3 MB/s eta 0:00:00 2024-03-04T17:41:37.016 INFO:teuthology.orchestra.run.smithi137.stdout:Collecting filelock<4,>=3.0.0 2024-03-04T17:41:37.056 INFO:teuthology.orchestra.run.smithi137.stdout: Downloading filelock-3.13.1-py3-none-any.whl (11 kB) 2024-03-04T17:41:37.172 INFO:teuthology.orchestra.run.smithi137.stdout:Collecting toml>=0.9.4 2024-03-04T17:41:37.214 INFO:teuthology.orchestra.run.smithi137.stdout: Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) 2024-03-04T17:41:37.352 INFO:teuthology.orchestra.run.smithi137.stdout:Collecting six<2,>=1.14.0 2024-03-04T17:41:37.395 INFO:teuthology.orchestra.run.smithi137.stdout: Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-03-04T17:41:37.592 INFO:teuthology.orchestra.run.smithi137.stdout:Collecting packaging>=14 2024-03-04T17:41:37.645 INFO:teuthology.orchestra.run.smithi137.stdout: Downloading packaging-23.2-py3-none-any.whl (53 kB) 2024-03-04T17:41:37.688 INFO:teuthology.orchestra.run.smithi137.stdout: ---------------------------------------- 53.0/53.0 KB 1.1 MB/s eta 0:00:00 2024-03-04T17:41:37.909 INFO:teuthology.orchestra.run.smithi137.stdout:Collecting platformdirs<5,>=3.9.1 2024-03-04T17:41:37.950 INFO:teuthology.orchestra.run.smithi137.stdout: Downloading platformdirs-4.2.0-py3-none-any.whl (17 kB) 2024-03-04T17:41:38.087 INFO:teuthology.orchestra.run.smithi137.stdout:Collecting distlib<1,>=0.3.7 2024-03-04T17:41:38.132 INFO:teuthology.orchestra.run.smithi137.stdout: Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB) 2024-03-04T17:41:38.240 INFO:teuthology.orchestra.run.smithi137.stdout: --------------------------------------- 468.9/468.9 KB 4.5 MB/s eta 0:00:00 2024-03-04T17:41:38.405 INFO:teuthology.orchestra.run.smithi137.stdout:Installing collected packages: distlib, toml, six, py, pluggy, platformdirs, packaging, filelock, virtualenv, tox 2024-03-04T17:41:39.415 INFO:teuthology.orchestra.run.smithi137.stdout:Successfully installed distlib-0.3.8 filelock-3.13.1 packaging-23.2 platformdirs-4.2.0 pluggy-0.13.1 py-1.11.0 six-1.16.0 toml-0.10.2 tox-3.15.0 virtualenv-20.25.1 2024-03-04T17:41:39.502 INFO:teuthology.run_tasks:Running task workunit... 2024-03-04T17:41:39.515 INFO:tasks.workunit:Pulling workunits from ref 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:41:39.516 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-04T17:41:39.519 DEBUG:teuthology.orchestra.run.smithi137:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:41:39.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:41:39.525 INFO:teuthology.orchestra.run.smithi137.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-04T17:41:39.525 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:41:39.571 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-04T17:41:39.572 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-04T17:41:39.619 INFO:tasks.workunit:timeout=3h 2024-03-04T17:41:39.620 INFO:tasks.workunit:cleanup=True 2024-03-04T17:41:39.620 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/pritha-srivastava/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:41:39.670 INFO:tasks.workunit.client.0.smithi137.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-04T17:42:41.326 INFO:tasks.workunit.client.0.smithi137.stderr:Note: switching to '3aec390336fdfdc8f927818b718cfb0361bd47c8'. 2024-03-04T17:42:41.326 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:42:41.326 INFO:tasks.workunit.client.0.smithi137.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T17:42:41.326 INFO:tasks.workunit.client.0.smithi137.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T17:42:41.326 INFO:tasks.workunit.client.0.smithi137.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T17:42:41.326 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:42:41.327 INFO:tasks.workunit.client.0.smithi137.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T17:42:41.327 INFO:tasks.workunit.client.0.smithi137.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T17:42:41.327 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:42:41.327 INFO:tasks.workunit.client.0.smithi137.stderr: git switch -c 2024-03-04T17:42:41.327 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:42:41.327 INFO:tasks.workunit.client.0.smithi137.stderr:Or undo this operation with: 2024-03-04T17:42:41.327 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:42:41.327 INFO:tasks.workunit.client.0.smithi137.stderr: git switch - 2024-03-04T17:42:41.327 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:42:41.327 INFO:tasks.workunit.client.0.smithi137.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T17:42:41.327 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:42:41.327 INFO:tasks.workunit.client.0.smithi137.stderr:HEAD is now at 3aec390336f qa/d4n: Start redis up before rgw 2024-03-04T17:42:41.332 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-04T17:42:41.380 INFO:tasks.workunit.client.0.smithi137.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T17:42:41.384 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T17:42:41.384 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T17:42:42.147 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T17:42:42.230 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T17:42:42.288 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T17:42:42.290 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T17:42:42.290 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T17:42:42.334 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T17:42:42.338 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:42:42.338 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-04T17:42:42.383 INFO:tasks.workunit:Running workunits matching rgw/run-datacache.sh on client.0... 2024-03-04T17:42:42.385 INFO:tasks.workunit:Running workunit rgw/run-datacache.sh... 2024-03-04T17:42:42.385 DEBUG:teuthology.orchestra.run.smithi137:workunit test rgw/run-datacache.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RGW_DATACACHE_PATH=/tmp/rgw_datacache adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-datacache.sh 2024-03-04T17:42:42.436 INFO:tasks.workunit.client.0.smithi137.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-datacache.sh 2024-03-04T17:42:42.437 INFO:tasks.workunit.client.0.smithi137.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T17:42:42.437 INFO:tasks.workunit.client.0.smithi137.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T17:42:45.566 INFO:tasks.workunit.client.0.smithi137.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-03-04T17:42:45.567 INFO:tasks.workunit.client.0.smithi137.stderr:++ deactivate nondestructive 2024-03-04T17:42:45.567 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n '' ']' 2024-03-04T17:42:45.567 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n '' ']' 2024-03-04T17:42:45.567 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T17:42:45.567 INFO:tasks.workunit.client.0.smithi137.stderr:++ hash -r 2024-03-04T17:42:45.567 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n '' ']' 2024-03-04T17:42:45.567 INFO:tasks.workunit.client.0.smithi137.stderr:++ unset VIRTUAL_ENV 2024-03-04T17:42:45.567 INFO:tasks.workunit.client.0.smithi137.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-03-04T17:42:45.567 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-03-04T17:42:45.567 INFO:tasks.workunit.client.0.smithi137.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T17:42:45.567 INFO:tasks.workunit.client.0.smithi137.stderr:++ export VIRTUAL_ENV 2024-03-04T17:42:45.567 INFO:tasks.workunit.client.0.smithi137.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T17:42:45.567 INFO:tasks.workunit.client.0.smithi137.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T17:42:45.568 INFO:tasks.workunit.client.0.smithi137.stderr:++ export PATH 2024-03-04T17:42:45.568 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n '' ']' 2024-03-04T17:42:45.568 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -z '' ']' 2024-03-04T17:42:45.568 INFO:tasks.workunit.client.0.smithi137.stderr:++ _OLD_VIRTUAL_PS1= 2024-03-04T17:42:45.568 INFO:tasks.workunit.client.0.smithi137.stderr:++ PS1='(rgw) ' 2024-03-04T17:42:45.568 INFO:tasks.workunit.client.0.smithi137.stderr:++ export PS1 2024-03-04T17:42:45.568 INFO:tasks.workunit.client.0.smithi137.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-03-04T17:42:45.568 INFO:tasks.workunit.client.0.smithi137.stderr:++ export VIRTUAL_ENV_PROMPT 2024-03-04T17:42:45.568 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T17:42:45.569 INFO:tasks.workunit.client.0.smithi137.stderr:++ hash -r 2024-03-04T17:42:45.569 INFO:tasks.workunit.client.0.smithi137.stderr:+ pip install pip --upgrade 2024-03-04T17:42:45.923 INFO:tasks.workunit.client.0.smithi137.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-03-04T17:42:46.320 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting pip 2024-03-04T17:42:46.521 INFO:tasks.workunit.client.0.smithi137.stdout: Downloading pip-24.0-py3-none-any.whl (2.1 MB) 2024-03-04T17:42:46.833 INFO:tasks.workunit.client.0.smithi137.stdout: ---------------------------------------- 2.1/2.1 MB 7.0 MB/s eta 0:00:00 2024-03-04T17:42:46.909 INFO:tasks.workunit.client.0.smithi137.stdout:Installing collected packages: pip 2024-03-04T17:42:46.909 INFO:tasks.workunit.client.0.smithi137.stdout: Attempting uninstall: pip 2024-03-04T17:42:46.910 INFO:tasks.workunit.client.0.smithi137.stdout: Found existing installation: pip 22.0.2 2024-03-04T17:42:47.197 INFO:tasks.workunit.client.0.smithi137.stdout: Uninstalling pip-22.0.2: 2024-03-04T17:42:47.204 INFO:tasks.workunit.client.0.smithi137.stdout: Successfully uninstalled pip-22.0.2 2024-03-04T17:42:48.252 INFO:tasks.workunit.client.0.smithi137.stdout:Successfully installed pip-24.0 2024-03-04T17:42:48.319 INFO:tasks.workunit.client.0.smithi137.stderr:+ pip install configobj 2024-03-04T17:42:49.035 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting configobj 2024-03-04T17:42:49.318 INFO:tasks.workunit.client.0.smithi137.stdout: Downloading configobj-5.0.8-py2.py3-none-any.whl.metadata (3.4 kB) 2024-03-04T17:42:49.528 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting six (from configobj) 2024-03-04T17:42:49.577 INFO:tasks.workunit.client.0.smithi137.stdout: Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-03-04T17:42:49.688 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading configobj-5.0.8-py2.py3-none-any.whl (36 kB) 2024-03-04T17:42:49.841 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-03-04T17:42:49.950 INFO:tasks.workunit.client.0.smithi137.stdout:Installing collected packages: six, configobj 2024-03-04T17:42:50.093 INFO:tasks.workunit.client.0.smithi137.stdout:Successfully installed configobj-5.0.8 six-1.16.0 2024-03-04T17:42:50.694 INFO:tasks.workunit.client.0.smithi137.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_datacache.py 2024-03-04T17:42:50.731 INFO:tasks.workunit.client.0.smithi137.stderr:DEBUG:root:datacache dir from config is: /tmp/rgw_datacache 2024-03-04T17:42:50.731 INFO:tasks.workunit.client.0.smithi137.stderr:DEBUG:root:exec_cmd(pwd) 2024-03-04T17:42:50.733 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:command succeeded 2024-03-04T17:42:50.733 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:b'/home/ubuntu/cephtest/mnt.0/client.0/tmp\n' 2024-03-04T17:42:50.733 INFO:tasks.workunit.client.0.smithi137.stderr:DEBUG:root:pwd is: /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:42:50.733 INFO:tasks.workunit.client.0.smithi137.stderr:DEBUG:root:exec_cmd(sudo netstat -nltp | egrep "rados|valgr") 2024-03-04T17:42:50.778 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:command succeeded 2024-03-04T17:42:50.778 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:b'tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 37075/valgrind.bin \ntcp6 0 0 :::443 :::* LISTEN 37075/valgrind.bin \n' 2024-03-04T17:42:50.778 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:radosgw port: 443 2024-03-04T17:42:50.778 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:radosgw endpoint is: 127.0.0.1 2024-03-04T17:42:50.778 INFO:tasks.workunit.client.0.smithi137.stderr:DEBUG:root:exec_cmd(dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat bs=1M count=7) 2024-03-04T17:42:50.809 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:command succeeded 2024-03-04T17:42:50.809 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:b'' 2024-03-04T17:42:50.809 INFO:tasks.workunit.client.0.smithi137.stderr:DEBUG:root:exec_cmd(radosgw-admin user create --uid rgw_datacache_user --display-name DatacacheUser --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn) 2024-03-04T17:42:51.008 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:command succeeded 2024-03-04T17:42:51.009 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:b'{\n "user_id": "rgw_datacache_user",\n "display_name": "DatacacheUser",\n "email": "",\n "suspended": 0,\n "max_buckets": 1000,\n "subusers": [],\n "keys": [\n {\n "user": "rgw_datacache_user",\n "access_key": "NX5QOQKC6BH2IDN8HC7A",\n "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn",\n "active": true\n }\n ],\n "swift_keys": [],\n "caps": [],\n "op_mask": "read, write, delete",\n "default_placement": "",\n "default_storage_class": "",\n "placement_tags": [],\n "bucket_quota": {\n "enabled": false,\n "check_on_raw": false,\n "max_size": -1,\n "max_size_kb": 0,\n "max_objects": -1\n },\n "user_quota": {\n "enabled": false,\n "check_on_raw": false,\n "max_size": -1,\n "max_size_kb": 0,\n "max_objects": -1\n },\n "temp_url_keys": [],\n "type": "rgw",\n "mfa_ids": []\n}\n\n' 2024-03-04T17:42:51.009 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:Creating s3cmd config... 2024-03-04T17:42:51.009 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:proto for s3cmd config is https 2024-03-04T17:42:51.009 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:s3cmd config written 2024-03-04T17:42:51.009 INFO:tasks.workunit.client.0.smithi137.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 mb s3://datacachebucket) 2024-03-04T17:42:52.040 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:command succeeded 2024-03-04T17:42:52.040 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:b"Bucket 's3://datacachebucket/' created\n" 2024-03-04T17:42:52.040 INFO:tasks.workunit.client.0.smithi137.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 put /home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat s3://datacachebucket) 2024-03-04T17:42:56.585 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:command succeeded 2024-03-04T17:42:56.585 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:b"upload: '/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat' -> 's3://datacachebucket/7M.dat' (7340032 bytes in 4.2 seconds, 1719.76 KB/s) [1 of 1]\n" 2024-03-04T17:42:56.585 INFO:tasks.workunit.client.0.smithi137.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 get s3://datacachebucket/7M.dat /home/ubuntu/cephtest/mnt.0/client.0/tmp/7M-get.dat --force) 2024-03-04T17:42:58.199 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:command succeeded 2024-03-04T17:42:58.199 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:b"download: 's3://datacachebucket/7M.dat' -> '/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M-get.dat' (7340032 bytes in 1.4 seconds, 5.12 MB/s)\n" 2024-03-04T17:42:58.199 INFO:tasks.workunit.client.0.smithi137.stderr:DEBUG:root:exec_cmd(radosgw-admin object stat --bucket=datacachebucket --object=7M.dat) 2024-03-04T17:42:58.590 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:command succeeded 2024-03-04T17:42:58.590 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:b'{\n "name": "7M.dat",\n "size": 7340376,\n "policy": {\n "acl": {\n "acl_user_map": [\n {\n "user": "rgw_datacache_user",\n "acl": 15\n }\n ],\n "acl_group_map": [],\n "grant_map": [\n {\n "id": "rgw_datacache_user",\n "grant": {\n "type": {\n "type": 0\n },\n "id": "rgw_datacache_user",\n "name": "DatacacheUser",\n "permission": {\n "flags": 15\n }\n }\n }\n ]\n },\n "owner": {\n "id": "rgw_datacache_user",\n "display_name": "DatacacheUser"\n }\n },\n "compression": {\n "compression_type": "snappy",\n "orig_size": 7340032,\n "blocks": [\n {\n "old_ofs": 0,\n "new_ofs": 0,\n "len": 4194500\n },\n {\n "old_ofs": 4194304,\n "new_ofs": 4194500,\n "len": 3145876\n }\n ]\n },\n "etag": "f36710ed3b7add1112bb470489bc4988",\n "tag": "f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4341982173111320208",\n "manifest": {\n "objs": [],\n "obj_size": 7340376,\n "explicit_objs": false,\n "head_size": 4194304,\n "max_head_size": 4194304,\n "prefix": ".ObTQiPAwi-w1lH3d3ArcqmOaQ6etcPb_",\n "rules": [\n {\n "key": 0,\n "val": {\n "start_part_num": 0,\n "start_ofs": 4194304,\n "part_size": 0,\n "stripe_max_size": 4194304,\n "override_prefix": ""\n }\n }\n ],\n "tail_instance": "",\n "tail_placement": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.1",\n "bucket_id": "f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "placement_rule": "default-placement"\n },\n "tier_type": "",\n "begin_iter": {\n "part_ofs": 0,\n "stripe_ofs": 0,\n "ofs": 0,\n "stripe_size": 4194304,\n "cur_part_id": 0,\n "cur_stripe": 0,\n "cur_override_prefix": "",\n "location": {\n "placement_rule": "default-placement",\n "obj": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.1",\n "bucket_id": "f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "key": {\n "name": "7M.dat",\n "instance": "",\n "ns": ""\n }\n },\n "raw_obj": {\n "pool": "",\n "oid": "",\n "loc": ""\n },\n "is_raw": false\n }\n },\n "end_iter": {\n "part_ofs": 4194304,\n "stripe_ofs": 4194304,\n "ofs": 7340376,\n "stripe_size": 3146072,\n "cur_part_id": 0,\n "cur_stripe": 1,\n "cur_override_prefix": "",\n "location": {\n "placement_rule": "default-placement",\n "obj": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.1",\n "bucket_id": "f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "key": {\n "name": ".ObTQiPAwi-w1lH3d3ArcqmOaQ6etcPb_1",\n "instance": "",\n "ns": "shadow"\n }\n },\n "raw_obj": {\n "pool": "",\n "oid": "",\n "loc": ""\n },\n "is_raw": false\n }\n }\n },\n "pg_ver": 0,\n "source_zone": 0,\n "torrent": "",\n "mtime": "2024-03-04T17:42:56.274267Z",\n "attrs": {\n "user.rgw.content_type": "application/octet-stream",\n "user.rgw.storage_class": "STANDARD",\n "user.rgw.tail_tag": "f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4341982173111320208",\n "user.rgw.x-amz-content-sha256": "d42a52ea5fc317eb82470be083b6a026fb83d830bc6509a065110e538145aa69",\n "user.rgw.x-amz-date": "20240304T174252Z",\n "user.rgw.x-amz-meta-s3cmd-attrs": "atime:1709574170/ctime:1709574170/gid:1271/gname:ubuntu/md5:f36710ed3b7add1112bb470489bc4988/mode:33204/mtime:1709574170/uid:1000/uname:ubuntu"\n }\n}\n' 2024-03-04T17:42:58.590 INFO:tasks.workunit.client.0.smithi137.stderr:DEBUG:root:Cached object name is: .ObTQiPAwi-w1lH3d3ArcqmOaQ6etcPb_ 2024-03-04T17:42:58.590 INFO:tasks.workunit.client.0.smithi137.stderr:DEBUG:root:exec_cmd(find /tmp/rgw_datacache -type f | wc -l) 2024-03-04T17:42:58.592 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:command succeeded 2024-03-04T17:42:58.592 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:b'0\n' 2024-03-04T17:42:58.592 INFO:tasks.workunit.client.0.smithi137.stderr:DEBUG:root:Check cache dir content: 0 2024-03-04T17:42:58.592 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:NOTICE: datacache test object not found, inspect if datacache was bypassed or disabled during this check. 2024-03-04T17:42:58.592 INFO:tasks.workunit.client.0.smithi137.stderr:INFO:root:Completed Datacache tests 2024-03-04T17:42:58.599 INFO:tasks.workunit.client.0.smithi137.stderr:+ deactivate 2024-03-04T17:42:58.599 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-03-04T17:42:58.599 INFO:tasks.workunit.client.0.smithi137.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T17:42:58.599 INFO:tasks.workunit.client.0.smithi137.stderr:+ export PATH 2024-03-04T17:42:58.600 INFO:tasks.workunit.client.0.smithi137.stderr:+ unset _OLD_VIRTUAL_PATH 2024-03-04T17:42:58.600 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' -n '' ']' 2024-03-04T17:42:58.600 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T17:42:58.600 INFO:tasks.workunit.client.0.smithi137.stderr:+ hash -r 2024-03-04T17:42:58.600 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' -n '' ']' 2024-03-04T17:42:58.600 INFO:tasks.workunit.client.0.smithi137.stderr:+ unset VIRTUAL_ENV 2024-03-04T17:42:58.600 INFO:tasks.workunit.client.0.smithi137.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-03-04T17:42:58.600 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' '!' '' = nondestructive ']' 2024-03-04T17:42:58.600 INFO:tasks.workunit.client.0.smithi137.stderr:+ unset -f deactivate 2024-03-04T17:42:58.600 INFO:tasks.workunit.client.0.smithi137.stdout:OK. 2024-03-04T17:42:58.601 INFO:tasks.workunit.client.0.smithi137.stderr:+ echo OK. 2024-03-04T17:42:58.601 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:42:58.601 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:42:58.659 INFO:tasks.workunit:Stopping ['rgw/run-datacache.sh'] on client.0... 2024-03-04T17:42:58.659 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-04T17:42:58.983 DEBUG:teuthology.parallel:result is None 2024-03-04T17:42:58.983 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:42:58.995 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:42:58.995 DEBUG:teuthology.orchestra.run.smithi137:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:42:59.073 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:42:59.074 INFO:teuthology.run_tasks:Running task workunit... 2024-03-04T17:42:59.085 INFO:tasks.workunit:Pulling workunits from ref 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:42:59.085 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-04T17:42:59.085 DEBUG:teuthology.orchestra.run.smithi137:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:42:59.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:42:59.119 INFO:teuthology.orchestra.run.smithi137.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-04T17:42:59.119 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:42:59.167 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-04T17:42:59.168 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-04T17:42:59.215 INFO:tasks.workunit:timeout=3h 2024-03-04T17:42:59.215 INFO:tasks.workunit:cleanup=True 2024-03-04T17:42:59.216 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/pritha-srivastava/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:42:59.265 INFO:tasks.workunit.client.0.smithi137.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-04T17:44:56.260 INFO:tasks.workunit.client.0.smithi137.stderr:Note: switching to '3aec390336fdfdc8f927818b718cfb0361bd47c8'. 2024-03-04T17:44:56.260 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:44:56.260 INFO:tasks.workunit.client.0.smithi137.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T17:44:56.260 INFO:tasks.workunit.client.0.smithi137.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T17:44:56.261 INFO:tasks.workunit.client.0.smithi137.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T17:44:56.261 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:44:56.261 INFO:tasks.workunit.client.0.smithi137.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T17:44:56.261 INFO:tasks.workunit.client.0.smithi137.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T17:44:56.261 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:44:56.261 INFO:tasks.workunit.client.0.smithi137.stderr: git switch -c 2024-03-04T17:44:56.261 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:44:56.261 INFO:tasks.workunit.client.0.smithi137.stderr:Or undo this operation with: 2024-03-04T17:44:56.261 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:44:56.261 INFO:tasks.workunit.client.0.smithi137.stderr: git switch - 2024-03-04T17:44:56.261 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:44:56.261 INFO:tasks.workunit.client.0.smithi137.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T17:44:56.261 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:44:56.262 INFO:tasks.workunit.client.0.smithi137.stderr:HEAD is now at 3aec390336f qa/d4n: Start redis up before rgw 2024-03-04T17:44:56.268 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-04T17:44:56.316 INFO:tasks.workunit.client.0.smithi137.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T17:44:56.319 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T17:44:56.319 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T17:44:56.367 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T17:44:56.415 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T17:44:56.453 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T17:44:56.454 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T17:44:56.454 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T17:44:56.500 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T17:44:56.503 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:44:56.503 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-04T17:44:56.551 INFO:tasks.workunit:Running workunits matching rgw/run-bucket-check.sh on client.0... 2024-03-04T17:44:56.552 INFO:tasks.workunit:Running workunit rgw/run-bucket-check.sh... 2024-03-04T17:44:56.552 DEBUG:teuthology.orchestra.run.smithi137:workunit test rgw/run-bucket-check.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2024-03-04T17:44:56.603 INFO:tasks.workunit.client.0.smithi137.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2024-03-04T17:44:56.604 INFO:tasks.workunit.client.0.smithi137.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T17:44:56.604 INFO:tasks.workunit.client.0.smithi137.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T17:44:59.698 INFO:tasks.workunit.client.0.smithi137.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-03-04T17:44:59.698 INFO:tasks.workunit.client.0.smithi137.stderr:++ deactivate nondestructive 2024-03-04T17:44:59.698 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n '' ']' 2024-03-04T17:44:59.698 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n '' ']' 2024-03-04T17:44:59.698 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T17:44:59.698 INFO:tasks.workunit.client.0.smithi137.stderr:++ hash -r 2024-03-04T17:44:59.698 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n '' ']' 2024-03-04T17:44:59.698 INFO:tasks.workunit.client.0.smithi137.stderr:++ unset VIRTUAL_ENV 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ export VIRTUAL_ENV 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ export PATH 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n '' ']' 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -z '' ']' 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ _OLD_VIRTUAL_PS1= 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ PS1='(rgw) ' 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ export PS1 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ export VIRTUAL_ENV_PROMPT 2024-03-04T17:44:59.699 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T17:44:59.700 INFO:tasks.workunit.client.0.smithi137.stderr:++ hash -r 2024-03-04T17:44:59.700 INFO:tasks.workunit.client.0.smithi137.stderr:+ pip install pip --upgrade 2024-03-04T17:45:00.064 INFO:tasks.workunit.client.0.smithi137.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-03-04T17:45:00.358 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting pip 2024-03-04T17:45:00.365 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-03-04T17:45:00.400 INFO:tasks.workunit.client.0.smithi137.stdout:Installing collected packages: pip 2024-03-04T17:45:00.400 INFO:tasks.workunit.client.0.smithi137.stdout: Attempting uninstall: pip 2024-03-04T17:45:00.401 INFO:tasks.workunit.client.0.smithi137.stdout: Found existing installation: pip 22.0.2 2024-03-04T17:45:00.660 INFO:tasks.workunit.client.0.smithi137.stdout: Uninstalling pip-22.0.2: 2024-03-04T17:45:00.666 INFO:tasks.workunit.client.0.smithi137.stdout: Successfully uninstalled pip-22.0.2 2024-03-04T17:45:01.741 INFO:tasks.workunit.client.0.smithi137.stdout:Successfully installed pip-24.0 2024-03-04T17:45:01.814 INFO:tasks.workunit.client.0.smithi137.stderr:+ pip install boto3 2024-03-04T17:45:02.967 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting boto3 2024-03-04T17:45:03.119 INFO:tasks.workunit.client.0.smithi137.stdout: Downloading boto3-1.34.54-py3-none-any.whl.metadata (6.6 kB) 2024-03-04T17:45:04.010 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting botocore<1.35.0,>=1.34.54 (from boto3) 2024-03-04T17:45:04.046 INFO:tasks.workunit.client.0.smithi137.stdout: Downloading botocore-1.34.54-py3-none-any.whl.metadata (5.7 kB) 2024-03-04T17:45:04.239 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-03-04T17:45:04.283 INFO:tasks.workunit.client.0.smithi137.stdout: Downloading jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-03-04T17:45:04.436 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-03-04T17:45:04.479 INFO:tasks.workunit.client.0.smithi137.stdout: Downloading s3transfer-0.10.0-py3-none-any.whl.metadata (1.7 kB) 2024-03-04T17:45:04.667 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T17:45:04.700 INFO:tasks.workunit.client.0.smithi137.stdout: Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-03-04T17:45:04.930 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting urllib3<2.1,>=1.25.4 (from botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T17:45:04.962 INFO:tasks.workunit.client.0.smithi137.stdout: Downloading urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB) 2024-03-04T17:45:05.357 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T17:45:05.358 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-03-04T17:45:05.407 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading boto3-1.34.54-py3-none-any.whl (139 kB) 2024-03-04T17:45:05.550 INFO:tasks.workunit.client.0.smithi137.stdout: --------------------------------------- 139.3/139.3 kB 969.4 kB/s eta 0:00:00 2024-03-04T17:45:05.600 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading botocore-1.34.54-py3-none-any.whl (12.0 MB) 2024-03-04T17:45:07.879 INFO:tasks.workunit.client.0.smithi137.stdout: ---------------------------------------- 12.0/12.0 MB 6.5 MB/s eta 0:00:00 2024-03-04T17:45:07.913 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-03-04T17:45:08.011 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading s3transfer-0.10.0-py3-none-any.whl (82 kB) 2024-03-04T17:45:08.101 INFO:tasks.workunit.client.0.smithi137.stdout: ---------------------------------------- 82.1/82.1 kB 852.7 kB/s eta 0:00:00 2024-03-04T17:45:08.131 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-03-04T17:45:08.207 INFO:tasks.workunit.client.0.smithi137.stdout: ---------------------------------------- 229.9/229.9 kB 3.0 MB/s eta 0:00:00 2024-03-04T17:45:08.245 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading urllib3-2.0.7-py3-none-any.whl (124 kB) 2024-03-04T17:45:08.301 INFO:tasks.workunit.client.0.smithi137.stdout: ---------------------------------------- 124.2/124.2 kB 2.2 MB/s eta 0:00:00 2024-03-04T17:45:08.305 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-03-04T17:45:08.454 INFO:tasks.workunit.client.0.smithi137.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-03-04T17:45:09.651 INFO:tasks.workunit.client.0.smithi137.stdout:Successfully installed boto3-1.34.54 botocore-1.34.54 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.0 six-1.16.0 urllib3-2.0.7 2024-03-04T17:45:09.800 INFO:tasks.workunit.client.0.smithi137.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_bucket_check.py 2024-03-04T17:45:09.940 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin user create --uid check-tester --display-name "Check Testing" --access-key OJODXSLNX4LUNHQG99PA --secret 3l6ffld34qaymfomuh832j94738aie2x4p2o8h6n 2024-03-04T17:45:09.977 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:09.977 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:10.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.015+0000 7f33e63c9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.019 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.019+0000 7f33e63c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.019 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.019+0000 7f33e63c9980 20 realm 2024-03-04T17:45:10.019 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.019+0000 7f33e63c9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.019+0000 7f33e63c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.019+0000 7f33e63c9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:10.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.019+0000 7f33e63c9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.019+0000 7f33e63c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.019+0000 7f33e63c9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.019+0000 7f33e63c9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:10.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.019+0000 7f33e63c9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.019+0000 7f33e63c9980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:10.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.019+0000 7f33e63c9980 20 searching for the correct realm 2024-03-04T17:45:10.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.023+0000 7f33e36e0640 0 -- 172.21.15.137:0/4102487986 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f33d4042c20 msgr2=0x7f33d406ea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:10.031 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.031+0000 7f33e36e0640 0 -- 172.21.15.137:0/4102487986 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x55d590fc02c0 msgr2=0x55d590fc26b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:10.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:10.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:10.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:10.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:10.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:10.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:10.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 zone default found 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 4 Realm: () 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 zonegroup default 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 started sync module instance, tier type = 2024-03-04T17:45:10.043 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.039+0000 7f33e63c9980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:10.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.067+0000 7f33e63c9980 20 add_watcher() i=0 2024-03-04T17:45:10.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.067+0000 7f33e63c9980 20 add_watcher() i=1 2024-03-04T17:45:10.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.067+0000 7f33e63c9980 20 add_watcher() i=2 2024-03-04T17:45:10.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.067+0000 7f33e63c9980 20 add_watcher() i=3 2024-03-04T17:45:10.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.067+0000 7f33e63c9980 20 add_watcher() i=4 2024-03-04T17:45:10.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.067+0000 7f33e63c9980 20 add_watcher() i=5 2024-03-04T17:45:10.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.067+0000 7f33e63c9980 20 add_watcher() i=6 2024-03-04T17:45:10.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.071+0000 7f33e63c9980 20 add_watcher() i=7 2024-03-04T17:45:10.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.071+0000 7f33e63c9980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:10.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.075+0000 7f3299fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:10.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.075+0000 7f33e63c9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:10.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.075+0000 7f33e63c9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:10.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 5 note: GC not initialized 2024-03-04T17:45:10.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f3298fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:10.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:10.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 20 Filter name: none 2024-03-04T17:45:10.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f32127fc640 20 reqs_thread_entry: start 2024-03-04T17:45:10.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 10 cache get: name=default.rgw.meta+users.uid+check-tester : miss 2024-03-04T17:45:10.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.080 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x0 2024-03-04T17:45:10.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 10 adding default.rgw.meta+users.uid+check-tester to cache LRU end 2024-03-04T17:45:10.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : miss 2024-03-04T17:45:10.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 20 rados->read ofs=0 len=0 2024-03-04T17:45:10.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:10.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x0 2024-03-04T17:45:10.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 10 adding default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-03-04T17:45:10.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2024-03-04T17:45:10.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2024-03-04T17:45:10.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2024-03-04T17:45:10.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 10 moving default.rgw.meta+users.uid+check-tester to cache LRU end 2024-03-04T17:45:10.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f33e63c9980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:check-tester, ofs0, ns] 2024-03-04T17:45:10.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f32c2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855361 cookie 94375748866096 notifier 4632 bl.length()=560 2024-03-04T17:45:10.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f32c2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2024-03-04T17:45:10.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.079+0000 7f32c2ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+check-tester to cache LRU end 2024-03-04T17:45:10.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.087+0000 7f33e63c9980 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2024-03-04T17:45:10.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.087+0000 7f33e63c9980 10 moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-03-04T17:45:10.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.087+0000 7f33e63c9980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.keys:OJODXSLNX4LUNHQG99PA, ofs0, ns] 2024-03-04T17:45:10.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.087+0000 7f32c2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855360 cookie 94375748834496 notifier 4632 bl.length()=186 2024-03-04T17:45:10.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.087+0000 7f32c2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2024-03-04T17:45:10.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.087+0000 7f32c2ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-03-04T17:45:10.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.087+0000 7f3299fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:10.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.091+0000 7f33e63c9980 20 remove_watcher() i=0 2024-03-04T17:45:10.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.091+0000 7f33e63c9980 2 removed watcher, disabling cache 2024-03-04T17:45:10.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.095+0000 7f33e63c9980 20 remove_watcher() i=1 2024-03-04T17:45:10.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.095+0000 7f33e63c9980 20 remove_watcher() i=2 2024-03-04T17:45:10.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.099+0000 7f33e63c9980 20 remove_watcher() i=3 2024-03-04T17:45:10.101 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.099+0000 7f33e63c9980 20 remove_watcher() i=4 2024-03-04T17:45:10.103 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.103+0000 7f33e63c9980 20 remove_watcher() i=5 2024-03-04T17:45:10.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.107+0000 7f33e63c9980 20 remove_watcher() i=6 2024-03-04T17:45:10.109 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:10.107+0000 7f33e63c9980 20 remove_watcher() i=7 2024-03-04T17:45:10.360 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:10.361 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:10.440 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:10.440 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:10.560 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:10.560 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:10.674 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:10.674 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:10.792 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:10.792 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:10.862 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:10.862 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:11.024 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:11.025 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:11.088 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:11.089 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:11.244 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:11.244 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:11.537 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:11.537 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:11.626 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:11.626 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:11.756 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:11.757 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:11.822 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:11.822 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:11.949 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:11.949 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:12.012 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:12.012 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:12.142 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:12.142 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:12.205 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: recalculated bucket check stats are correct 2024-03-04T17:45:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:45:12.206 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2024-03-04T17:45:12.240 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:12.240 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:12.284 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.283+0000 7fa7cd652980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.283+0000 7fa7cd652980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.283+0000 7fa7cd652980 20 realm 2024-03-04T17:45:12.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.283+0000 7fa7cd652980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.287+0000 7fa7cd652980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.287+0000 7fa7cd652980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:12.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.287+0000 7fa7cd652980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.287+0000 7fa7cd652980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.287+0000 7fa7cd652980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.287+0000 7fa7cd652980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:12.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.287+0000 7fa7cd652980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.287+0000 7fa7cd652980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:12.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.287+0000 7fa7cd652980 20 searching for the correct realm 2024-03-04T17:45:12.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.291+0000 7fa7cb96b640 0 -- 172.21.15.137:0/2685416769 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fa7c4008040 msgr2=0x7fa7c4036430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:12.507 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.507+0000 7fa7cb96b640 0 -- 172.21.15.137:0/2685416769 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x558a44691f90 msgr2=0x558a44694450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:12.517 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.515+0000 7fa7cd652980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:12.517 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.515+0000 7fa7cd652980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:12.517 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.515+0000 7fa7cd652980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:12.517 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.515+0000 7fa7cd652980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:12.517 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.515+0000 7fa7cd652980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:12.517 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.515+0000 7fa7cd652980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:12.518 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.515+0000 7fa7cd652980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.518 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.515+0000 7fa7cd652980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.518 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.515+0000 7fa7cd652980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.518 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.515+0000 7fa7cd652980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:12.518 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.515+0000 7fa7cd652980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:12.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 20 zone default found 2024-03-04T17:45:12.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 4 Realm: () 2024-03-04T17:45:12.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:12.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:12.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:12.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 20 zonegroup default 2024-03-04T17:45:12.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 20 started sync module instance, tier type = 2024-03-04T17:45:12.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.519+0000 7fa7cd652980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:12.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.551+0000 7fa7cd652980 20 add_watcher() i=0 2024-03-04T17:45:12.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.551+0000 7fa7cd652980 20 add_watcher() i=1 2024-03-04T17:45:12.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.551+0000 7fa7cd652980 20 add_watcher() i=2 2024-03-04T17:45:12.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.551+0000 7fa7cd652980 20 add_watcher() i=3 2024-03-04T17:45:12.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.551+0000 7fa7cd652980 20 add_watcher() i=4 2024-03-04T17:45:12.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.551+0000 7fa7cd652980 20 add_watcher() i=5 2024-03-04T17:45:12.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.551+0000 7fa7cd652980 20 add_watcher() i=6 2024-03-04T17:45:12.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.555+0000 7fa7cd652980 20 add_watcher() i=7 2024-03-04T17:45:12.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.555+0000 7fa7cd652980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:12.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.559+0000 7fa6827e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:12.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.559+0000 7fa7cd652980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:12.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.559+0000 7fa7cd652980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:12.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.559+0000 7fa7cd652980 5 note: GC not initialized 2024-03-04T17:45:12.561 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.559+0000 7fa6817e2640 20 reqs_thread_entry: start 2024-03-04T17:45:12.561 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.559+0000 7fa7cd652980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:12.561 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.559+0000 7fa7cd652980 20 Filter name: none 2024-03-04T17:45:12.561 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.559+0000 7fa67a7fc640 20 reqs_thread_entry: start 2024-03-04T17:45:12.561 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.559+0000 7fa7cd652980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:12.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.559+0000 7fa7cd652980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:12.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:12.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:12.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 : miss 2024-03-04T17:45:12.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:12.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 info.flags=0x17 2024-03-04T17:45:12.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 to cache LRU end 2024-03-04T17:45:12.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:12.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:12.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 2024-03-04T17:45:12.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 20 list_objects_ordered: starting attempt 1 2024-03-04T17:45:12.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-03-04T17:45:12.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T17:45:12.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T17:45:12.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T17:45:12.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.563+0000 7fa7cd652980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T17:45:12.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.571+0000 7fa6827e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:12.577 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.575+0000 7fa7cd652980 20 remove_watcher() i=0 2024-03-04T17:45:12.577 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.575+0000 7fa7cd652980 2 removed watcher, disabling cache 2024-03-04T17:45:12.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.579+0000 7fa7cd652980 20 remove_watcher() i=1 2024-03-04T17:45:12.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.583+0000 7fa7cd652980 20 remove_watcher() i=2 2024-03-04T17:45:12.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.583+0000 7fa7cd652980 20 remove_watcher() i=3 2024-03-04T17:45:12.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.587+0000 7fa7cd652980 20 remove_watcher() i=4 2024-03-04T17:45:12.590 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.587+0000 7fa7cd652980 20 remove_watcher() i=5 2024-03-04T17:45:12.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.591+0000 7fa7cd652980 20 remove_watcher() i=6 2024-03-04T17:45:12.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.595+0000 7fa7cd652980 20 remove_watcher() i=7 2024-03-04T17:45:12.612 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket check-bucket 2024-03-04T17:45:12.642 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:12.642 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:12.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.683+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.687+0000 7f57def70980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.687+0000 7f57def70980 20 realm 2024-03-04T17:45:12.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.687+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.687+0000 7f57def70980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.687+0000 7f57def70980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:12.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.687+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.687+0000 7f57def70980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.687+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.687+0000 7f57def70980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:12.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.687+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.691 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.691+0000 7f57def70980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:12.691 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.691+0000 7f57def70980 20 searching for the correct realm 2024-03-04T17:45:12.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.695+0000 7f57dd289640 0 -- 172.21.15.137:0/3391508773 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f57d40142a0 msgr2=0x7f57d408f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:12.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.911+0000 7f57dd289640 0 -- 172.21.15.137:0/3391508773 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x56448ba78840 msgr2=0x56448ba7acd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:12.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.919+0000 7f57def70980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:12.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.919+0000 7f57def70980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:12.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.919+0000 7f57def70980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:12.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.919+0000 7f57def70980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:12.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.919+0000 7f57def70980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:12.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.919+0000 7f57def70980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:12.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.919+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.919+0000 7f57def70980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.919+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:12.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:12.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 20 zone default found 2024-03-04T17:45:12.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 4 Realm: () 2024-03-04T17:45:12.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:12.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:12.924 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:12.924 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 20 zonegroup default 2024-03-04T17:45:12.924 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.924 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.924 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.925 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.925 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.926 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.926 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 20 started sync module instance, tier type = 2024-03-04T17:45:12.926 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.923+0000 7f57def70980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:12.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.951+0000 7f57def70980 20 add_watcher() i=0 2024-03-04T17:45:12.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.951+0000 7f57def70980 20 add_watcher() i=1 2024-03-04T17:45:12.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.951+0000 7f57def70980 20 add_watcher() i=2 2024-03-04T17:45:12.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.951+0000 7f57def70980 20 add_watcher() i=3 2024-03-04T17:45:12.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.951+0000 7f57def70980 20 add_watcher() i=4 2024-03-04T17:45:12.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.951+0000 7f57def70980 20 add_watcher() i=5 2024-03-04T17:45:12.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.951+0000 7f57def70980 20 add_watcher() i=6 2024-03-04T17:45:12.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.955+0000 7f57def70980 20 add_watcher() i=7 2024-03-04T17:45:12.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.955+0000 7f57def70980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:12.960 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.959+0000 7f5692fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:12.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.959+0000 7f57def70980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:12.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.959+0000 7f57def70980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:12.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.959+0000 7f57def70980 5 note: GC not initialized 2024-03-04T17:45:12.962 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.959+0000 7f5691fe3640 20 reqs_thread_entry: start 2024-03-04T17:45:12.962 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.959+0000 7f57def70980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:12.962 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.959+0000 7f57def70980 20 Filter name: none 2024-03-04T17:45:12.962 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.959+0000 7f56037fe640 20 reqs_thread_entry: start 2024-03-04T17:45:12.962 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.963+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.963 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.963+0000 7f57def70980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:12.963 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.963+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.964 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.963+0000 7f57def70980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:12.964 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.963+0000 7f57def70980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:12.965 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.963+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.965 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.963+0000 7f57def70980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:12.965 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.963+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.966 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.963+0000 7f57def70980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:12.966 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.963+0000 7f57def70980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:12.967 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.967+0000 7f57def70980 20 rados->read ofs=0 len=0 2024-03-04T17:45:12.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.967+0000 7f57def70980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:12.969 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.967+0000 7f5692fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:12.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.971+0000 7f57def70980 20 remove_watcher() i=0 2024-03-04T17:45:12.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.971+0000 7f57def70980 2 removed watcher, disabling cache 2024-03-04T17:45:12.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.975+0000 7f57def70980 20 remove_watcher() i=1 2024-03-04T17:45:12.978 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.979+0000 7f57def70980 20 remove_watcher() i=2 2024-03-04T17:45:12.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.979+0000 7f57def70980 20 remove_watcher() i=3 2024-03-04T17:45:12.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.983+0000 7f57def70980 20 remove_watcher() i=4 2024-03-04T17:45:12.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.987+0000 7f57def70980 20 remove_watcher() i=5 2024-03-04T17:45:12.989 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.987+0000 7f57def70980 20 remove_watcher() i=6 2024-03-04T17:45:12.992 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:12.991+0000 7f57def70980 20 remove_watcher() i=7 2024-03-04T17:45:13.007 INFO:tasks.workunit.client.0.smithi137.stderr:{'rgw.main': {'size': 54, 'size_actual': 24576, 'size_utilized': 65, 'size_kb': 1, 'size_kb_actual': 24, 'size_kb_utilized': 1, 'num_objects': 6}} 2024-03-04T17:45:13.007 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: bucket check unlinked does not report normal entries 2024-03-04T17:45:13.007 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:45:13.007 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-03-04T17:45:13.041 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:13.041 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:13.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.079+0000 7f67fec8f980 20 rados->read ofs=0 len=0 2024-03-04T17:45:13.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.083+0000 7f67fec8f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:13.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.083+0000 7f67fec8f980 20 realm 2024-03-04T17:45:13.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.083+0000 7f67fec8f980 20 rados->read ofs=0 len=0 2024-03-04T17:45:13.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.083+0000 7f67fec8f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:13.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.083+0000 7f67fec8f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:13.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.083+0000 7f67fec8f980 20 rados->read ofs=0 len=0 2024-03-04T17:45:13.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.083+0000 7f67fec8f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:13.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.083+0000 7f67fec8f980 20 rados->read ofs=0 len=0 2024-03-04T17:45:13.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.083+0000 7f67fec8f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:13.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.083+0000 7f67fec8f980 20 rados->read ofs=0 len=0 2024-03-04T17:45:13.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.087+0000 7f67fec8f980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:13.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.087+0000 7f67fec8f980 20 searching for the correct realm 2024-03-04T17:45:13.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.091+0000 7f67fcfa8640 0 -- 172.21.15.137:0/3770105229 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x5624628ecd80 msgr2=0x5624628ef240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:13.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.095+0000 7f67fcfa8640 0 -- 172.21.15.137:0/3770105229 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x5624628e0570 msgr2=0x5624628e29e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:13.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:13.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:13.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:13.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:13.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:13.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:13.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 rados->read ofs=0 len=0 2024-03-04T17:45:13.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:13.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 rados->read ofs=0 len=0 2024-03-04T17:45:13.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:13.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 rados->read ofs=0 len=0 2024-03-04T17:45:13.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:13.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 zone default found 2024-03-04T17:45:13.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 4 Realm: () 2024-03-04T17:45:13.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:13.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:13.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:13.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 zonegroup default 2024-03-04T17:45:13.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 rados->read ofs=0 len=0 2024-03-04T17:45:13.109 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:13.109 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 rados->read ofs=0 len=0 2024-03-04T17:45:13.109 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:13.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 rados->read ofs=0 len=0 2024-03-04T17:45:13.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:13.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 started sync module instance, tier type = 2024-03-04T17:45:13.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.107+0000 7f67fec8f980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:13.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.131+0000 7f67fec8f980 20 add_watcher() i=0 2024-03-04T17:45:13.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.131+0000 7f67fec8f980 20 add_watcher() i=1 2024-03-04T17:45:13.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.131+0000 7f67fec8f980 20 add_watcher() i=2 2024-03-04T17:45:13.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.131+0000 7f67fec8f980 20 add_watcher() i=3 2024-03-04T17:45:13.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.131+0000 7f67fec8f980 20 add_watcher() i=4 2024-03-04T17:45:13.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.131+0000 7f67fec8f980 20 add_watcher() i=5 2024-03-04T17:45:13.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.131+0000 7f67fec8f980 20 add_watcher() i=6 2024-03-04T17:45:13.133 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.131+0000 7f67fec8f980 20 add_watcher() i=7 2024-03-04T17:45:13.133 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.131+0000 7f67fec8f980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:13.136 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.135+0000 7f66b2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:13.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.135+0000 7f67fec8f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:13.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.135+0000 7f67fec8f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:13.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.135+0000 7f67fec8f980 5 note: GC not initialized 2024-03-04T17:45:13.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.135+0000 7f66b1fe3640 20 reqs_thread_entry: start 2024-03-04T17:45:13.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.135+0000 7f67fec8f980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:13.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.135+0000 7f67fec8f980 20 Filter name: none 2024-03-04T17:45:13.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.135+0000 7f66ab7fe640 20 reqs_thread_entry: start 2024-03-04T17:45:13.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.135+0000 7f67fec8f980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:13.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.135+0000 7f67fec8f980 20 rados->read ofs=0 len=0 2024-03-04T17:45:13.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.139+0000 7f67fec8f980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:13.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.139+0000 7f67fec8f980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:13.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.139+0000 7f67fec8f980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:13.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.139+0000 7f67fec8f980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 : miss 2024-03-04T17:45:13.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.139+0000 7f67fec8f980 20 rados->read ofs=0 len=0 2024-03-04T17:45:13.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.139+0000 7f67fec8f980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:13.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.139+0000 7f67fec8f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 info.flags=0x17 2024-03-04T17:45:13.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.139+0000 7f67fec8f980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 to cache LRU end 2024-03-04T17:45:13.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.139+0000 7f67fec8f980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:13.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.139+0000 7f67fec8f980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:13.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.139+0000 7f67fec8f980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 2024-03-04T17:45:13.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.139+0000 7f67fec8f980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:13.141 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.139+0000 7f67fec8f980 1 NOTICE: finished shard 0 (0 entries found) 2024-03-04T17:45:13.141 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.139+0000 7f67fec8f980 1 NOTICE: finished all shards (0 entries found) 2024-03-04T17:45:13.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.143+0000 7f66b2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:13.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.147+0000 7f67fec8f980 20 remove_watcher() i=0 2024-03-04T17:45:13.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.147+0000 7f67fec8f980 2 removed watcher, disabling cache 2024-03-04T17:45:13.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.147+0000 7f67fec8f980 20 remove_watcher() i=1 2024-03-04T17:45:13.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.151+0000 7f67fec8f980 20 remove_watcher() i=2 2024-03-04T17:45:13.153 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.151+0000 7f67fec8f980 20 remove_watcher() i=3 2024-03-04T17:45:13.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.155+0000 7f67fec8f980 20 remove_watcher() i=4 2024-03-04T17:45:13.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.155+0000 7f67fec8f980 20 remove_watcher() i=5 2024-03-04T17:45:13.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.159+0000 7f67fec8f980 20 remove_watcher() i=6 2024-03-04T17:45:13.163 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:13.163+0000 7f67fec8f980 20 remove_watcher() i=7 2024-03-04T17:45:13.177 INFO:tasks.workunit.client.0.smithi137.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-03-04T17:45:13.586 INFO:tasks.workunit.client.0.smithi137.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-03-04T17:45:14.980 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:14.980 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:15.135 INFO:tasks.workunit.client.0.smithi137.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:15.136 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-03-04T17:45:15.171 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:15.171 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:15.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.211+0000 7f7c47e07980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.216 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.215+0000 7f7c47e07980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.216 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.215+0000 7f7c47e07980 20 realm 2024-03-04T17:45:15.216 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.215+0000 7f7c47e07980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.216 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.215+0000 7f7c47e07980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.216 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.215+0000 7f7c47e07980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:15.216 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.215+0000 7f7c47e07980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.216 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.215+0000 7f7c47e07980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.216 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.215+0000 7f7c47e07980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.217 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.215+0000 7f7c47e07980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:15.217 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.215+0000 7f7c47e07980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.218 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.215+0000 7f7c47e07980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:15.218 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.215+0000 7f7c47e07980 20 searching for the correct realm 2024-03-04T17:45:15.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.223+0000 7f7c46120640 0 -- 172.21.15.137:0/105311982 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f7c4002d920 msgr2=0x7f7c4000e420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:15.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.439+0000 7f7c46120640 0 -- 172.21.15.137:0/105311982 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55fbd4af3a60 msgr2=0x55fbd4af5f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:15.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.447+0000 7f7c47e07980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:15.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.447+0000 7f7c47e07980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:15.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.447+0000 7f7c47e07980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:15.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.447+0000 7f7c47e07980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:15.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.447+0000 7f7c47e07980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:15.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.447+0000 7f7c47e07980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:15.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.447+0000 7f7c47e07980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.451 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.451 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.451 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:15.451 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.452 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:15.452 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 20 zone default found 2024-03-04T17:45:15.452 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 4 Realm: () 2024-03-04T17:45:15.452 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:15.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:15.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:15.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 20 zonegroup default 2024-03-04T17:45:15.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.454 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.454 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.454 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.451+0000 7f7c47e07980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.455+0000 7f7c47e07980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.455+0000 7f7c47e07980 20 started sync module instance, tier type = 2024-03-04T17:45:15.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.455+0000 7f7c47e07980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:15.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.479+0000 7f7c47e07980 20 add_watcher() i=0 2024-03-04T17:45:15.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.479+0000 7f7c47e07980 20 add_watcher() i=1 2024-03-04T17:45:15.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.479+0000 7f7c47e07980 20 add_watcher() i=2 2024-03-04T17:45:15.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.479+0000 7f7c47e07980 20 add_watcher() i=3 2024-03-04T17:45:15.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.479+0000 7f7c47e07980 20 add_watcher() i=4 2024-03-04T17:45:15.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.479+0000 7f7c47e07980 20 add_watcher() i=5 2024-03-04T17:45:15.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.479+0000 7f7c47e07980 20 add_watcher() i=6 2024-03-04T17:45:15.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.483+0000 7f7c47e07980 20 add_watcher() i=7 2024-03-04T17:45:15.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.483+0000 7f7c47e07980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:15.488 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.487+0000 7f7af77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:15.489 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.487+0000 7f7c47e07980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:15.489 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.487+0000 7f7c47e07980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:15.489 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.487+0000 7f7c47e07980 5 note: GC not initialized 2024-03-04T17:45:15.490 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.487+0000 7f7af67fc640 20 reqs_thread_entry: start 2024-03-04T17:45:15.490 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.487+0000 7f7c47e07980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:15.490 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.491+0000 7f7c47e07980 20 Filter name: none 2024-03-04T17:45:15.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.491+0000 7f7a67fff640 20 reqs_thread_entry: start 2024-03-04T17:45:15.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.491+0000 7f7c47e07980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.491+0000 7f7c47e07980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:15.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.491+0000 7f7c47e07980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.493 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.491+0000 7f7c47e07980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:15.493 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.491+0000 7f7c47e07980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:15.493 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.491+0000 7f7c47e07980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-03-04T17:45:15.493 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.491+0000 7f7c47e07980 20 INFO: main: starting shard=0 2024-03-04T17:45:15.493 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.491+0000 7f7c47e07980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:15.494 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.495+0000 7f7c47e07980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-03-04T17:45:15.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.495+0000 7f7c47e07980 20 INFO: main: done 2024-03-04T17:45:15.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.495+0000 7f7af77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:15.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.499+0000 7f7c47e07980 20 remove_watcher() i=0 2024-03-04T17:45:15.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.499+0000 7f7c47e07980 2 removed watcher, disabling cache 2024-03-04T17:45:15.503 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.503+0000 7f7c47e07980 20 remove_watcher() i=1 2024-03-04T17:45:15.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.503+0000 7f7c47e07980 20 remove_watcher() i=2 2024-03-04T17:45:15.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.507+0000 7f7c47e07980 20 remove_watcher() i=3 2024-03-04T17:45:15.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.511+0000 7f7c47e07980 20 remove_watcher() i=4 2024-03-04T17:45:15.513 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.511+0000 7f7c47e07980 20 remove_watcher() i=5 2024-03-04T17:45:15.516 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.515+0000 7f7c47e07980 20 remove_watcher() i=6 2024-03-04T17:45:15.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.519+0000 7f7c47e07980 20 remove_watcher() i=7 2024-03-04T17:45:15.543 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:15.543 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:15.576 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:15.576 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:15.735 INFO:tasks.workunit.client.0.smithi137.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:15.736 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-03-04T17:45:15.770 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:15.770 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:15.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.811+0000 7f8b6a0dc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.814 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.815+0000 7f8b6a0dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.815+0000 7f8b6a0dc980 20 realm 2024-03-04T17:45:15.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.815+0000 7f8b6a0dc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.815+0000 7f8b6a0dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.815+0000 7f8b6a0dc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:15.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.815+0000 7f8b6a0dc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.816 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.815+0000 7f8b6a0dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.816 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.815+0000 7f8b6a0dc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.816 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.815+0000 7f8b6a0dc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:15.816 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.815+0000 7f8b6a0dc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.817 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.815+0000 7f8b6a0dc980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:15.817 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.815+0000 7f8b6a0dc980 20 searching for the correct realm 2024-03-04T17:45:15.822 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.823+0000 7f8b683f5640 0 -- 172.21.15.137:0/1752416992 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x56042b11bf20 msgr2=0x56042b11e3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:15.828 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.827+0000 7f8b673f3640 0 -- 172.21.15.137:0/1752416992 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x56042b117280 msgr2=0x56042b119670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:15.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:15.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:15.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:15.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:15.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:15.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:15.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:15.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:15.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 zone default found 2024-03-04T17:45:15.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 4 Realm: () 2024-03-04T17:45:15.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:15.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:15.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:15.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 zonegroup default 2024-03-04T17:45:15.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.835+0000 7f8b6a0dc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.839+0000 7f8b6a0dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.839+0000 7f8b6a0dc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.839+0000 7f8b6a0dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.839+0000 7f8b6a0dc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.839+0000 7f8b6a0dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:15.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.839+0000 7f8b6a0dc980 20 started sync module instance, tier type = 2024-03-04T17:45:15.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.839+0000 7f8b6a0dc980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:15.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.859+0000 7f8b6a0dc980 20 add_watcher() i=0 2024-03-04T17:45:15.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.859+0000 7f8b6a0dc980 20 add_watcher() i=1 2024-03-04T17:45:15.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.859+0000 7f8b6a0dc980 20 add_watcher() i=2 2024-03-04T17:45:15.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.859+0000 7f8b6a0dc980 20 add_watcher() i=3 2024-03-04T17:45:15.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.859+0000 7f8b6a0dc980 20 add_watcher() i=4 2024-03-04T17:45:15.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.859+0000 7f8b6a0dc980 20 add_watcher() i=5 2024-03-04T17:45:15.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.859+0000 7f8b6a0dc980 20 add_watcher() i=6 2024-03-04T17:45:15.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.863+0000 7f8b6a0dc980 20 add_watcher() i=7 2024-03-04T17:45:15.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.863+0000 7f8b6a0dc980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:15.868 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.867+0000 7f8a1dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:15.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.867+0000 7f8b6a0dc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:15.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.867+0000 7f8b6a0dc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:15.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.867+0000 7f8b6a0dc980 5 note: GC not initialized 2024-03-04T17:45:15.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.867+0000 7f8a1cfe1640 20 reqs_thread_entry: start 2024-03-04T17:45:15.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.867+0000 7f8b6a0dc980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:15.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.867+0000 7f8b6a0dc980 20 Filter name: none 2024-03-04T17:45:15.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.867+0000 7f89967fc640 20 reqs_thread_entry: start 2024-03-04T17:45:15.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.867+0000 7f8b6a0dc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.871+0000 7f8b6a0dc980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:15.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.871+0000 7f8b6a0dc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:15.871 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.871+0000 7f8b6a0dc980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:15.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.871+0000 7f8b6a0dc980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:15.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.871+0000 7f8b6a0dc980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-03-04T17:45:15.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.871+0000 7f8b6a0dc980 20 INFO: main: starting shard=0 2024-03-04T17:45:15.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.871+0000 7f8b6a0dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:15.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.871+0000 7f8b6a0dc980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-03-04T17:45:15.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.871+0000 7f8b6a0dc980 20 INFO: main: done 2024-03-04T17:45:15.874 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.875+0000 7f8a1dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:15.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.879+0000 7f8b6a0dc980 20 remove_watcher() i=0 2024-03-04T17:45:15.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.879+0000 7f8b6a0dc980 2 removed watcher, disabling cache 2024-03-04T17:45:15.880 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.879+0000 7f8b6a0dc980 20 remove_watcher() i=1 2024-03-04T17:45:15.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.883+0000 7f8b6a0dc980 20 remove_watcher() i=2 2024-03-04T17:45:15.885 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.883+0000 7f8b6a0dc980 20 remove_watcher() i=3 2024-03-04T17:45:15.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.887+0000 7f8b6a0dc980 20 remove_watcher() i=4 2024-03-04T17:45:15.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.887+0000 7f8b6a0dc980 20 remove_watcher() i=5 2024-03-04T17:45:15.893 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.891+0000 7f8b6a0dc980 20 remove_watcher() i=6 2024-03-04T17:45:15.896 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:15.895+0000 7f8b6a0dc980 20 remove_watcher() i=7 2024-03-04T17:45:15.911 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:15.912 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:15.936 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:15.936 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:15.964 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:15.964 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:16.491 INFO:tasks.workunit.client.0.smithi137.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:16.492 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket check-bucket --object e 2024-03-04T17:45:16.526 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:16.526 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:16.570 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.567+0000 7f644e362980 20 rados->read ofs=0 len=0 2024-03-04T17:45:16.776 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.775+0000 7f644e362980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:16.776 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.775+0000 7f644e362980 20 realm 2024-03-04T17:45:16.776 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.775+0000 7f644e362980 20 rados->read ofs=0 len=0 2024-03-04T17:45:16.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.775+0000 7f644e362980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:16.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.775+0000 7f644e362980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:16.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.775+0000 7f644e362980 20 rados->read ofs=0 len=0 2024-03-04T17:45:16.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.775+0000 7f644e362980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:16.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.775+0000 7f644e362980 20 rados->read ofs=0 len=0 2024-03-04T17:45:16.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.779+0000 7f644e362980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:16.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.779+0000 7f644e362980 20 rados->read ofs=0 len=0 2024-03-04T17:45:16.781 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.779+0000 7f644e362980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:16.781 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.779+0000 7f644e362980 20 searching for the correct realm 2024-03-04T17:45:16.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.783+0000 7f644be7a640 0 -- 172.21.15.137:0/1973590855 >> [v2:172.21.15.137:6808/3491386063,v1:172.21.15.137:6810/3491386063] conn(0x7f643c02d060 msgr2=0x7f643c057430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:16.997 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:16.995+0000 7f644b679640 0 -- 172.21.15.137:0/1973590855 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x55771b534450 msgr2=0x55771b5368e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:17.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.011+0000 7f644e362980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:17.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.011+0000 7f644e362980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:17.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.011+0000 7f644e362980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:17.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.011+0000 7f644e362980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:17.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.011+0000 7f644e362980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:17.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.011+0000 7f644e362980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:17.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.011+0000 7f644e362980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.015 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.015+0000 7f644e362980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.015 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.015+0000 7f644e362980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.015 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.015+0000 7f644e362980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:17.015 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.015+0000 7f644e362980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.015+0000 7f644e362980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:17.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.015+0000 7f644e362980 20 zone default found 2024-03-04T17:45:17.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.015+0000 7f644e362980 4 Realm: () 2024-03-04T17:45:17.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.015+0000 7f644e362980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:17.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.015+0000 7f644e362980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:17.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.015+0000 7f644e362980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:17.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.015+0000 7f644e362980 20 zonegroup default 2024-03-04T17:45:17.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.015+0000 7f644e362980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.018 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.015+0000 7f644e362980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.018 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.015+0000 7f644e362980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.018 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.019+0000 7f644e362980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.018 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.019+0000 7f644e362980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.019 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.019+0000 7f644e362980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.019 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.019+0000 7f644e362980 20 started sync module instance, tier type = 2024-03-04T17:45:17.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.019+0000 7f644e362980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:17.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.047+0000 7f644e362980 20 add_watcher() i=0 2024-03-04T17:45:17.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.047+0000 7f644e362980 20 add_watcher() i=1 2024-03-04T17:45:17.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.047+0000 7f644e362980 20 add_watcher() i=2 2024-03-04T17:45:17.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.047+0000 7f644e362980 20 add_watcher() i=3 2024-03-04T17:45:17.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.047+0000 7f644e362980 20 add_watcher() i=4 2024-03-04T17:45:17.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.047+0000 7f644e362980 20 add_watcher() i=5 2024-03-04T17:45:17.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.047+0000 7f644e362980 20 add_watcher() i=6 2024-03-04T17:45:17.052 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.051+0000 7f644e362980 20 add_watcher() i=7 2024-03-04T17:45:17.052 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.051+0000 7f644e362980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:17.056 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.055+0000 7f6301fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:17.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.055+0000 7f644e362980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:17.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.055+0000 7f644e362980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:17.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.055+0000 7f644e362980 5 note: GC not initialized 2024-03-04T17:45:17.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.055+0000 7f6300fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:17.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.055+0000 7f644e362980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:17.059 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.059+0000 7f644e362980 20 Filter name: none 2024-03-04T17:45:17.059 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.059+0000 7f627a7fc640 20 reqs_thread_entry: start 2024-03-04T17:45:17.059 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.059+0000 7f644e362980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.060 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.059+0000 7f644e362980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:17.060 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.059+0000 7f644e362980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.059+0000 7f644e362980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:17.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.059+0000 7f644e362980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:17.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.059+0000 7f644e362980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-03-04T17:45:17.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.059+0000 7f644e362980 20 INFO: main: starting shard=0 2024-03-04T17:45:17.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.059+0000 7f644e362980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:17.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.063+0000 7f644e362980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T17:45:17.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.063+0000 7f644e362980 20 INFO: main: done 2024-03-04T17:45:17.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.063+0000 7f6301fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:17.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.067+0000 7f644e362980 20 remove_watcher() i=0 2024-03-04T17:45:17.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.067+0000 7f644e362980 2 removed watcher, disabling cache 2024-03-04T17:45:17.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.071+0000 7f644e362980 20 remove_watcher() i=1 2024-03-04T17:45:17.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.075+0000 7f644e362980 20 remove_watcher() i=2 2024-03-04T17:45:17.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.075+0000 7f644e362980 20 remove_watcher() i=3 2024-03-04T17:45:17.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.079+0000 7f644e362980 20 remove_watcher() i=4 2024-03-04T17:45:17.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.083+0000 7f644e362980 20 remove_watcher() i=5 2024-03-04T17:45:17.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.083+0000 7f644e362980 20 remove_watcher() i=6 2024-03-04T17:45:17.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.087+0000 7f644e362980 20 remove_watcher() i=7 2024-03-04T17:45:17.111 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:17.111 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:17.139 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:17.139 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:17.235 INFO:tasks.workunit.client.0.smithi137.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:17.236 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket check-bucket --object f 2024-03-04T17:45:17.269 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:17.269 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:17.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.311+0000 7f32c6765980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.314 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.315+0000 7f32c6765980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.314 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.315+0000 7f32c6765980 20 realm 2024-03-04T17:45:17.315 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.315+0000 7f32c6765980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.315 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.315+0000 7f32c6765980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.315 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.315+0000 7f32c6765980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:17.315 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.315+0000 7f32c6765980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.315+0000 7f32c6765980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.315+0000 7f32c6765980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.315+0000 7f32c6765980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:17.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.315+0000 7f32c6765980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.317 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.315+0000 7f32c6765980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:17.317 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.315+0000 7f32c6765980 20 searching for the correct realm 2024-03-04T17:45:17.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.319+0000 7f32c3a7c640 0 -- 172.21.15.137:0/3797812313 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f32b40fa090 msgr2=0x7f32b41102e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:17.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.531+0000 7f32c3a7c640 0 -- 172.21.15.137:0/3797812313 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x558dd07a0590 msgr2=0x558dd07a2980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:17.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.547+0000 7f32c6765980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:17.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.547+0000 7f32c6765980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:17.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.547+0000 7f32c6765980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:17.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.547+0000 7f32c6765980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:17.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.547+0000 7f32c6765980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:17.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.547+0000 7f32c6765980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:17.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.547+0000 7f32c6765980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.547+0000 7f32c6765980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.547+0000 7f32c6765980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.547+0000 7f32c6765980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:17.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.547+0000 7f32c6765980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:17.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 20 zone default found 2024-03-04T17:45:17.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 4 Realm: () 2024-03-04T17:45:17.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:17.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:17.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:17.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 20 zonegroup default 2024-03-04T17:45:17.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:17.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 20 started sync module instance, tier type = 2024-03-04T17:45:17.553 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.551+0000 7f32c6765980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:17.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.579+0000 7f32c6765980 20 add_watcher() i=0 2024-03-04T17:45:17.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.579+0000 7f32c6765980 20 add_watcher() i=1 2024-03-04T17:45:17.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.579+0000 7f32c6765980 20 add_watcher() i=2 2024-03-04T17:45:17.580 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.579+0000 7f32c6765980 20 add_watcher() i=3 2024-03-04T17:45:17.580 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.579+0000 7f32c6765980 20 add_watcher() i=4 2024-03-04T17:45:17.580 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.579+0000 7f32c6765980 20 add_watcher() i=5 2024-03-04T17:45:17.580 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.579+0000 7f32c6765980 20 add_watcher() i=6 2024-03-04T17:45:17.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.583+0000 7f32c6765980 20 add_watcher() i=7 2024-03-04T17:45:17.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.583+0000 7f32c6765980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:17.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.587+0000 7f317afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:17.588 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.587+0000 7f32c6765980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:17.588 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.587+0000 7f32c6765980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:17.588 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.587+0000 7f32c6765980 5 note: GC not initialized 2024-03-04T17:45:17.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.587+0000 7f3179fdb640 20 reqs_thread_entry: start 2024-03-04T17:45:17.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.587+0000 7f32c6765980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:17.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.587+0000 7f32c6765980 20 Filter name: none 2024-03-04T17:45:17.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.587+0000 7f30eb7fe640 20 reqs_thread_entry: start 2024-03-04T17:45:17.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.587+0000 7f32c6765980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.590 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.591+0000 7f32c6765980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:17.590 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.591+0000 7f32c6765980 20 rados->read ofs=0 len=0 2024-03-04T17:45:17.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.591+0000 7f32c6765980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:17.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.591+0000 7f32c6765980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:17.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.591+0000 7f32c6765980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-03-04T17:45:17.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.591+0000 7f32c6765980 20 INFO: main: starting shard=0 2024-03-04T17:45:17.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.591+0000 7f32c6765980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:17.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.591+0000 7f32c6765980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T17:45:17.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.591+0000 7f32c6765980 20 INFO: main: done 2024-03-04T17:45:17.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.595+0000 7f317afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:17.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.599+0000 7f32c6765980 20 remove_watcher() i=0 2024-03-04T17:45:17.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.599+0000 7f32c6765980 2 removed watcher, disabling cache 2024-03-04T17:45:17.601 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.599+0000 7f32c6765980 20 remove_watcher() i=1 2024-03-04T17:45:17.604 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.603+0000 7f32c6765980 20 remove_watcher() i=2 2024-03-04T17:45:17.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.603+0000 7f32c6765980 20 remove_watcher() i=3 2024-03-04T17:45:17.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.607+0000 7f32c6765980 20 remove_watcher() i=4 2024-03-04T17:45:17.610 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.611+0000 7f32c6765980 20 remove_watcher() i=5 2024-03-04T17:45:17.613 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.611+0000 7f32c6765980 20 remove_watcher() i=6 2024-03-04T17:45:17.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:17.615+0000 7f32c6765980 20 remove_watcher() i=7 2024-03-04T17:45:17.638 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:17.638 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:17.677 INFO:tasks.workunit.client.0.smithi137.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-03-04T17:45:18.098 INFO:tasks.workunit.client.0.smithi137.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-03-04T17:45:18.518 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: bucket check unlinked finds unlistable entries 2024-03-04T17:45:18.518 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:45:18.518 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-03-04T17:45:18.552 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:18.552 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:18.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.591+0000 7f6f6a2ae980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.595+0000 7f6f6a2ae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.595+0000 7f6f6a2ae980 20 realm 2024-03-04T17:45:18.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.595+0000 7f6f6a2ae980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.595+0000 7f6f6a2ae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.595+0000 7f6f6a2ae980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:18.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.595+0000 7f6f6a2ae980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.595+0000 7f6f6a2ae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.595+0000 7f6f6a2ae980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.595+0000 7f6f6a2ae980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:18.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.595+0000 7f6f6a2ae980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.595+0000 7f6f6a2ae980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:18.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.595+0000 7f6f6a2ae980 20 searching for the correct realm 2024-03-04T17:45:18.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.599+0000 7f6f685c7640 0 -- 172.21.15.137:0/1246708212 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f6f60017910 msgr2=0x7f6f600323f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:18.816 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.815+0000 7f6f685c7640 0 -- 172.21.15.137:0/1246708212 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55f1feaf10d0 msgr2=0x55f1feaf3590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:18.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.823+0000 7f6f6a2ae980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:18.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.823+0000 7f6f6a2ae980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:18.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.823+0000 7f6f6a2ae980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:18.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.823+0000 7f6f6a2ae980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:18.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.823+0000 7f6f6a2ae980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:18.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.823+0000 7f6f6a2ae980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:18.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.823+0000 7f6f6a2ae980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.826 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.823+0000 7f6f6a2ae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.826 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.823+0000 7f6f6a2ae980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.826 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:18.826 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.827 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:18.827 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 20 zone default found 2024-03-04T17:45:18.827 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 4 Realm: () 2024-03-04T17:45:18.827 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:18.827 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:18.827 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:18.827 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 20 zonegroup default 2024-03-04T17:45:18.827 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.828 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.828 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.828 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.829 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.829 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:18.829 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 20 started sync module instance, tier type = 2024-03-04T17:45:18.830 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.827+0000 7f6f6a2ae980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:18.853 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.851+0000 7f6f6a2ae980 20 add_watcher() i=0 2024-03-04T17:45:18.853 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.851+0000 7f6f6a2ae980 20 add_watcher() i=1 2024-03-04T17:45:18.853 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.851+0000 7f6f6a2ae980 20 add_watcher() i=2 2024-03-04T17:45:18.853 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.851+0000 7f6f6a2ae980 20 add_watcher() i=3 2024-03-04T17:45:18.853 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.851+0000 7f6f6a2ae980 20 add_watcher() i=4 2024-03-04T17:45:18.853 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.851+0000 7f6f6a2ae980 20 add_watcher() i=5 2024-03-04T17:45:18.853 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.851+0000 7f6f6a2ae980 20 add_watcher() i=6 2024-03-04T17:45:18.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.855+0000 7f6f6a2ae980 20 add_watcher() i=7 2024-03-04T17:45:18.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.855+0000 7f6f6a2ae980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:18.860 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.859+0000 7f6e1dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:18.861 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.859+0000 7f6f6a2ae980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:18.861 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.859+0000 7f6f6a2ae980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:18.861 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.859+0000 7f6f6a2ae980 5 note: GC not initialized 2024-03-04T17:45:18.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.859+0000 7f6e1cfe1640 20 reqs_thread_entry: start 2024-03-04T17:45:18.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:18.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 20 Filter name: none 2024-03-04T17:45:18.863 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6d967fc640 20 reqs_thread_entry: start 2024-03-04T17:45:18.863 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:18.863 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.864 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:18.864 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:18.864 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:18.864 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 : miss 2024-03-04T17:45:18.864 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 20 rados->read ofs=0 len=0 2024-03-04T17:45:18.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:18.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 info.flags=0x17 2024-03-04T17:45:18.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 to cache LRU end 2024-03-04T17:45:18.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:18.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:18.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 2024-03-04T17:45:18.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.863+0000 7f6f6a2ae980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:18.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.871+0000 7f6f6a2ae980 1 NOTICE: finished shard 0 (4 entries found) 2024-03-04T17:45:18.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.871+0000 7f6f6a2ae980 1 NOTICE: finished all shards (4 entries found) 2024-03-04T17:45:18.874 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.875+0000 7f6e1dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:18.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.879+0000 7f6f6a2ae980 20 remove_watcher() i=0 2024-03-04T17:45:18.880 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.879+0000 7f6f6a2ae980 2 removed watcher, disabling cache 2024-03-04T17:45:18.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.879+0000 7f6f6a2ae980 20 remove_watcher() i=1 2024-03-04T17:45:18.884 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.883+0000 7f6f6a2ae980 20 remove_watcher() i=2 2024-03-04T17:45:18.887 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.887+0000 7f6f6a2ae980 20 remove_watcher() i=3 2024-03-04T17:45:18.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.887+0000 7f6f6a2ae980 20 remove_watcher() i=4 2024-03-04T17:45:18.891 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.891+0000 7f6f6a2ae980 20 remove_watcher() i=5 2024-03-04T17:45:18.894 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.891+0000 7f6f6a2ae980 20 remove_watcher() i=6 2024-03-04T17:45:18.897 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:18.895+0000 7f6f6a2ae980 20 remove_watcher() i=7 2024-03-04T17:45:18.911 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: unlinked entries are not listable 2024-03-04T17:45:18.911 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:45:18.914 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:18.914 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:19.031 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: GET returns 404 for unlinked entry keys that have no other versions 2024-03-04T17:45:19.031 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:45:19.037 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:19.037 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:19.140 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:19.140 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:19.235 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: bucket check unlinked fixes unlistable entries 2024-03-04T17:45:19.235 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:45:19.236 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-03-04T17:45:19.275 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:19.275 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:19.318 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.319+0000 7ff73bc4d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.319+0000 7ff73bc4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:19.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.319+0000 7ff73bc4d980 20 realm 2024-03-04T17:45:19.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.319+0000 7ff73bc4d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.319+0000 7ff73bc4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:19.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.319+0000 7ff73bc4d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:19.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.319+0000 7ff73bc4d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.319+0000 7ff73bc4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:19.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.319+0000 7ff73bc4d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.323+0000 7ff73bc4d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:19.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.323+0000 7ff73bc4d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.323+0000 7ff73bc4d980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:19.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.323+0000 7ff73bc4d980 20 searching for the correct realm 2024-03-04T17:45:19.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.327+0000 7ff739f66640 0 -- 172.21.15.137:0/1956751532 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7ff734018070 msgr2=0x7ff73401da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:19.544 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.543+0000 7ff739f66640 0 -- 172.21.15.137:0/1956751532 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55e8c48ba410 msgr2=0x55e8c48bc8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:19.554 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.551+0000 7ff73bc4d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:19.554 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.551+0000 7ff73bc4d980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:19.554 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.551+0000 7ff73bc4d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:19.554 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.551+0000 7ff73bc4d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:19.554 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.551+0000 7ff73bc4d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:19.554 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.551+0000 7ff73bc4d980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:19.554 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.551+0000 7ff73bc4d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:19.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:19.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:19.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 20 zone default found 2024-03-04T17:45:19.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 4 Realm: () 2024-03-04T17:45:19.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:19.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:19.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:19.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 20 zonegroup default 2024-03-04T17:45:19.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:19.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:19.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.555+0000 7ff73bc4d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.559+0000 7ff73bc4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:19.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.559+0000 7ff73bc4d980 20 started sync module instance, tier type = 2024-03-04T17:45:19.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.559+0000 7ff73bc4d980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:19.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.579+0000 7ff73bc4d980 20 add_watcher() i=0 2024-03-04T17:45:19.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.579+0000 7ff73bc4d980 20 add_watcher() i=1 2024-03-04T17:45:19.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.579+0000 7ff73bc4d980 20 add_watcher() i=2 2024-03-04T17:45:19.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.579+0000 7ff73bc4d980 20 add_watcher() i=3 2024-03-04T17:45:19.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.579+0000 7ff73bc4d980 20 add_watcher() i=4 2024-03-04T17:45:19.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.579+0000 7ff73bc4d980 20 add_watcher() i=5 2024-03-04T17:45:19.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.579+0000 7ff73bc4d980 20 add_watcher() i=6 2024-03-04T17:45:19.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.583+0000 7ff73bc4d980 20 add_watcher() i=7 2024-03-04T17:45:19.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.583+0000 7ff73bc4d980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:19.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.587+0000 7ff5ef7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:19.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.587+0000 7ff73bc4d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:19.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.587+0000 7ff73bc4d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:19.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.587+0000 7ff73bc4d980 5 note: GC not initialized 2024-03-04T17:45:19.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.587+0000 7ff5ee7e4640 20 reqs_thread_entry: start 2024-03-04T17:45:19.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.587+0000 7ff73bc4d980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:19.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.587+0000 7ff73bc4d980 20 Filter name: none 2024-03-04T17:45:19.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff55ffff640 20 reqs_thread_entry: start 2024-03-04T17:45:19.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff73bc4d980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:19.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff73bc4d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff73bc4d980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:19.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff73bc4d980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:19.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff73bc4d980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:19.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff73bc4d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 : miss 2024-03-04T17:45:19.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff73bc4d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:19.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff73bc4d980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:19.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff73bc4d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 info.flags=0x17 2024-03-04T17:45:19.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff73bc4d980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 to cache LRU end 2024-03-04T17:45:19.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff73bc4d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:19.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff73bc4d980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:19.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff73bc4d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 2024-03-04T17:45:19.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.591+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.595+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.595+0000 7ff73bc4d980 20 get_obj_state: octx=0x55e8c4641310 obj=check-bucket:c state=0x55e8c48ec708 s->prefetch_data=0 2024-03-04T17:45:19.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.595+0000 7ff73bc4d980 20 get_obj_state: setting s->obj_tag to ygkw4jg0cs9h6691yi9ghvw830ty70xg 2024-03-04T17:45:19.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.595+0000 7ff73bc4d980 20 get_obj_state_impl: setting s->olh_tag to ixksrzjdt0pwg5sfjlxsvb2gx4rtzjij 2024-03-04T17:45:19.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.595+0000 7ff73bc4d980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=ixksrzjdt0pwg5sfjlxsvb2gx4rtzjij 2024-03-04T17:45:19.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.599+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.601 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.599+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.603+0000 7ff73bc4d980 20 olh_log_entry: epoch=3 op=3 key=c[5qPvAxK4JptXRBW3oRfvc95ZA2yuvTX] 2024-03-04T17:45:19.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.603+0000 7ff73bc4d980 20 get_obj_state: octx=0x55e8c4641310 obj=check-bucket:_:5qPvAxK4JptXRBW3oRfvc95ZA2yuvTX_c state=0x55e8c48eb8a8 s->prefetch_data=0 2024-03-04T17:45:19.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.603+0000 7ff73bc4d980 10 manifest: total_size = 11 2024-03-04T17:45:19.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.603+0000 7ff73bc4d980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.6760560727966469369 2024-03-04T17:45:19.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.603+0000 7ff73bc4d980 20 get_obj_state: octx=0x55e8c4641310 obj=check-bucket:_:5qPvAxK4JptXRBW3oRfvc95ZA2yuvTX_c state=0x55e8c48eb8a8 s->prefetch_data=0 2024-03-04T17:45:19.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.603+0000 7ff73bc4d980 20 prepare_atomic_modification: state is not atomic. state=0x55e8c48eb8a8 2024-03-04T17:45:19.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.603+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.611+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.611+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.611+0000 7ff60bfff640 20 handle_completion(): completion ok for obj=c[5qPvAxK4JptXRBW3oRfvc95ZA2yuvTX] 2024-03-04T17:45:19.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.615+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.615+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.617 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.615+0000 7ff73bc4d980 20 get_obj_state: octx=0x55e8c4641310 obj=check-bucket:d state=0x55e8c48ec708 s->prefetch_data=0 2024-03-04T17:45:19.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.619+0000 7ff73bc4d980 20 get_obj_state: setting s->obj_tag to 9oxbb493sgz3f175eea8xx4zb2ogf397 2024-03-04T17:45:19.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.619+0000 7ff73bc4d980 20 get_obj_state_impl: setting s->olh_tag to fvoe12fhqv9p23a3c58sza5pfp2o2jne 2024-03-04T17:45:19.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.619+0000 7ff73bc4d980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=fvoe12fhqv9p23a3c58sza5pfp2o2jne 2024-03-04T17:45:19.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.619+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.625 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.623+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.626 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.623+0000 7ff73bc4d980 20 olh_log_entry: epoch=3 op=3 key=d[eYn2fsg4o3ZrmHCn0-VvJskS8-HgRhV] 2024-03-04T17:45:19.626 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.623+0000 7ff73bc4d980 20 get_obj_state: octx=0x55e8c4641310 obj=check-bucket:_:eYn2fsg4o3ZrmHCn0-VvJskS8-HgRhV_d state=0x55e8c48e7868 s->prefetch_data=0 2024-03-04T17:45:19.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.627+0000 7ff73bc4d980 10 manifest: total_size = 22 2024-03-04T17:45:19.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.627+0000 7ff73bc4d980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2597710389582714402 2024-03-04T17:45:19.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.627+0000 7ff73bc4d980 20 get_obj_state: octx=0x55e8c4641310 obj=check-bucket:_:eYn2fsg4o3ZrmHCn0-VvJskS8-HgRhV_d state=0x55e8c48e7868 s->prefetch_data=0 2024-03-04T17:45:19.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.627+0000 7ff73bc4d980 20 prepare_atomic_modification: state is not atomic. state=0x55e8c48e7868 2024-03-04T17:45:19.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.627+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.636 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.635+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.636 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.635+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.637 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.635+0000 7ff61cff9640 20 handle_completion(): completion ok for obj=d[eYn2fsg4o3ZrmHCn0-VvJskS8-HgRhV] 2024-03-04T17:45:19.640 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.639+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.639+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.639+0000 7ff73bc4d980 20 get_obj_state: octx=0x55e8c4641310 obj=check-bucket:e state=0x55e8c48ec708 s->prefetch_data=0 2024-03-04T17:45:19.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.643+0000 7ff73bc4d980 20 get_obj_state: setting s->obj_tag to 82y5hq8y08bss3s34pg44ys2q7is1yc3 2024-03-04T17:45:19.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.643+0000 7ff73bc4d980 20 get_obj_state_impl: setting s->olh_tag to atgpan9u92hg54kx2hijg1c0mazec167 2024-03-04T17:45:19.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.643+0000 7ff73bc4d980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=atgpan9u92hg54kx2hijg1c0mazec167 2024-03-04T17:45:19.646 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.643+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.649 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.647+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.651+0000 7ff73bc4d980 20 olh_log_entry: epoch=2 op=2 key=e[] 2024-03-04T17:45:19.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.651+0000 7ff73bc4d980 20 olh_log_entry: epoch=2 op=3 key=e[YXYwWTWm.uhH9zLgkvljgbn3SrJ3dqV] 2024-03-04T17:45:19.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.651+0000 7ff73bc4d980 20 get_obj_state: octx=0x55e8c4641310 obj=check-bucket:_:YXYwWTWm.uhH9zLgkvljgbn3SrJ3dqV_e state=0x55e8c48e7868 s->prefetch_data=0 2024-03-04T17:45:19.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.651+0000 7ff73bc4d980 10 manifest: total_size = 22 2024-03-04T17:45:19.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.651+0000 7ff73bc4d980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.6185869379401710798 2024-03-04T17:45:19.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.651+0000 7ff73bc4d980 20 get_obj_state: octx=0x55e8c4641310 obj=check-bucket:_:YXYwWTWm.uhH9zLgkvljgbn3SrJ3dqV_e state=0x55e8c48e7868 s->prefetch_data=0 2024-03-04T17:45:19.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.651+0000 7ff73bc4d980 20 prepare_atomic_modification: state is not atomic. state=0x55e8c48e7868 2024-03-04T17:45:19.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.651+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.661 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.659+0000 7ff61cff9640 20 handle_completion(): completion ok for obj=e[YXYwWTWm.uhH9zLgkvljgbn3SrJ3dqV] 2024-03-04T17:45:19.661 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.659+0000 7ff73bc4d980 20 get_obj_state: octx=0x55e8c4641310 obj=check-bucket:e state=0x55e8c48ec708 s->prefetch_data=0 2024-03-04T17:45:19.661 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.659+0000 7ff73bc4d980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=atgpan9u92hg54kx2hijg1c0mazec167 2024-03-04T17:45:19.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.667+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.671+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.675+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.675 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.675+0000 7ff73bc4d980 20 get_obj_state: octx=0x55e8c4641310 obj=check-bucket:f state=0x55e8c491d388 s->prefetch_data=0 2024-03-04T17:45:19.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.675+0000 7ff73bc4d980 20 get_obj_state: setting s->obj_tag to emb3psfs2c2xui38qvgxp5ws2v32lwdk 2024-03-04T17:45:19.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.675+0000 7ff73bc4d980 20 get_obj_state_impl: setting s->olh_tag to w05ncownjommluw0g5lyhw7wqblsoamh 2024-03-04T17:45:19.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.675+0000 7ff73bc4d980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=w05ncownjommluw0g5lyhw7wqblsoamh 2024-03-04T17:45:19.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.679+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.683+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.683+0000 7ff73bc4d980 20 olh_log_entry: epoch=2 op=2 key=f[] 2024-03-04T17:45:19.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.683+0000 7ff73bc4d980 20 olh_log_entry: epoch=2 op=3 key=f[9NCqBi0T.JJVNBuzLWSeauBQXJC3SBt] 2024-03-04T17:45:19.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.683+0000 7ff73bc4d980 20 get_obj_state: octx=0x55e8c4641310 obj=check-bucket:_:9NCqBi0T.JJVNBuzLWSeauBQXJC3SBt_f state=0x55e8c48e7868 s->prefetch_data=0 2024-03-04T17:45:19.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.683+0000 7ff73bc4d980 10 manifest: total_size = 22 2024-03-04T17:45:19.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.683+0000 7ff73bc4d980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8445870960246440544 2024-03-04T17:45:19.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.683+0000 7ff73bc4d980 20 get_obj_state: octx=0x55e8c4641310 obj=check-bucket:_:9NCqBi0T.JJVNBuzLWSeauBQXJC3SBt_f state=0x55e8c48e7868 s->prefetch_data=0 2024-03-04T17:45:19.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.683+0000 7ff73bc4d980 20 prepare_atomic_modification: state is not atomic. state=0x55e8c48e7868 2024-03-04T17:45:19.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.683+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.695+0000 7ff60bfff640 20 handle_completion(): completion ok for obj=f[9NCqBi0T.JJVNBuzLWSeauBQXJC3SBt] 2024-03-04T17:45:19.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.695+0000 7ff73bc4d980 20 get_obj_state: octx=0x55e8c4641310 obj=check-bucket:f state=0x55e8c491d388 s->prefetch_data=0 2024-03-04T17:45:19.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.695+0000 7ff73bc4d980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=w05ncownjommluw0g5lyhw7wqblsoamh 2024-03-04T17:45:19.703 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.703+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.707+0000 7ff73bc4d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:19.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.707+0000 7ff73bc4d980 1 NOTICE: finished shard 0 (4 entries removed) 2024-03-04T17:45:19.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.707+0000 7ff73bc4d980 1 NOTICE: finished all shards (4 entries removed) 2024-03-04T17:45:19.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.707+0000 7ff5ef7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:19.714 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.711+0000 7ff73bc4d980 20 remove_watcher() i=0 2024-03-04T17:45:19.714 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.711+0000 7ff73bc4d980 2 removed watcher, disabling cache 2024-03-04T17:45:19.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.715+0000 7ff73bc4d980 20 remove_watcher() i=1 2024-03-04T17:45:19.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.719+0000 7ff73bc4d980 20 remove_watcher() i=2 2024-03-04T17:45:19.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.723+0000 7ff73bc4d980 20 remove_watcher() i=3 2024-03-04T17:45:19.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.723+0000 7ff73bc4d980 20 remove_watcher() i=4 2024-03-04T17:45:19.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.727+0000 7ff73bc4d980 20 remove_watcher() i=5 2024-03-04T17:45:19.729 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.727+0000 7ff73bc4d980 20 remove_watcher() i=6 2024-03-04T17:45:19.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:19.731+0000 7ff73bc4d980 20 remove_watcher() i=7 2024-03-04T17:45:19.752 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:19.752 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:19.779 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:19.779 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:19.822 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:19.823 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:19.873 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:19.873 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:19.919 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: bucket check unlinked does not affect normal entries 2024-03-04T17:45:19.919 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:45:19.923 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:19.923 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:20.030 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.030 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:20.068 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.068 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:20.131 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.131 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:20.192 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.192 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:20.245 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: bucket check unlinked does not find new unlistable entries after fix 2024-03-04T17:45:20.245 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:45:20.246 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-03-04T17:45:20.284 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:20.284 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:20.327 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.327+0000 7fc729cde980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.327+0000 7fc729cde980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.327+0000 7fc729cde980 20 realm 2024-03-04T17:45:20.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.327+0000 7fc729cde980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.331+0000 7fc729cde980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.331+0000 7fc729cde980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:20.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.331+0000 7fc729cde980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.331+0000 7fc729cde980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.331+0000 7fc729cde980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.331+0000 7fc729cde980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:20.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.331+0000 7fc729cde980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.331+0000 7fc729cde980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:20.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.331+0000 7fc729cde980 20 searching for the correct realm 2024-03-04T17:45:20.335 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.335+0000 7fc727ff7640 0 -- 172.21.15.137:0/1629270113 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x55e1bd0e59b0 msgr2=0x55e1bd0e7da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:20.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.343+0000 7fc7277f6640 0 -- 172.21.15.137:0/1629270113 >> [v2:172.21.15.137:6824/1039931256,v1:172.21.15.137:6825/1039931256] conn(0x55e1bd0e9b80 msgr2=0x55e1bd0ebfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:20.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.351+0000 7fc729cde980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:20.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.351+0000 7fc729cde980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:20.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.351+0000 7fc729cde980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:20.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.351+0000 7fc729cde980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:20.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.351+0000 7fc729cde980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:20.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.351+0000 7fc729cde980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:20.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.351+0000 7fc729cde980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:20.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:20.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 zone default found 2024-03-04T17:45:20.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 4 Realm: () 2024-03-04T17:45:20.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:20.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:20.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:20.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 zonegroup default 2024-03-04T17:45:20.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 started sync module instance, tier type = 2024-03-04T17:45:20.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.355+0000 7fc729cde980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:20.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.379+0000 7fc729cde980 20 add_watcher() i=0 2024-03-04T17:45:20.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.379+0000 7fc729cde980 20 add_watcher() i=1 2024-03-04T17:45:20.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.379+0000 7fc729cde980 20 add_watcher() i=2 2024-03-04T17:45:20.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.379+0000 7fc729cde980 20 add_watcher() i=3 2024-03-04T17:45:20.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.379+0000 7fc729cde980 20 add_watcher() i=4 2024-03-04T17:45:20.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.379+0000 7fc729cde980 20 add_watcher() i=5 2024-03-04T17:45:20.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.379+0000 7fc729cde980 20 add_watcher() i=6 2024-03-04T17:45:20.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.379+0000 7fc729cde980 20 add_watcher() i=7 2024-03-04T17:45:20.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.379+0000 7fc729cde980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:20.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.383+0000 7fc5ddfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:20.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.383+0000 7fc729cde980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:20.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.383+0000 7fc729cde980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:20.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.383+0000 7fc729cde980 5 note: GC not initialized 2024-03-04T17:45:20.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc5dcfe1640 20 reqs_thread_entry: start 2024-03-04T17:45:20.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:20.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 20 Filter name: none 2024-03-04T17:45:20.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc5567fc640 20 reqs_thread_entry: start 2024-03-04T17:45:20.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:20.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:20.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:20.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:20.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 : miss 2024-03-04T17:45:20.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:20.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 info.flags=0x17 2024-03-04T17:45:20.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 to cache LRU end 2024-03-04T17:45:20.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:20.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:20.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 2024-03-04T17:45:20.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:20.390 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 1 NOTICE: finished shard 0 (0 entries found) 2024-03-04T17:45:20.390 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.387+0000 7fc729cde980 1 NOTICE: finished all shards (0 entries found) 2024-03-04T17:45:20.390 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.391+0000 7fc5ddfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:20.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.395+0000 7fc729cde980 20 remove_watcher() i=0 2024-03-04T17:45:20.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.395+0000 7fc729cde980 2 removed watcher, disabling cache 2024-03-04T17:45:20.397 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.395+0000 7fc729cde980 20 remove_watcher() i=1 2024-03-04T17:45:20.399 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.399+0000 7fc729cde980 20 remove_watcher() i=2 2024-03-04T17:45:20.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.399+0000 7fc729cde980 20 remove_watcher() i=3 2024-03-04T17:45:20.403 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.403+0000 7fc729cde980 20 remove_watcher() i=4 2024-03-04T17:45:20.405 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.403+0000 7fc729cde980 20 remove_watcher() i=5 2024-03-04T17:45:20.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.407+0000 7fc729cde980 20 remove_watcher() i=6 2024-03-04T17:45:20.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.411+0000 7fc729cde980 20 remove_watcher() i=7 2024-03-04T17:45:20.427 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.428 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:20.666 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:20.666 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:20.772 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-03-04T17:45:20.806 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:20.806 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:20.850 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.847+0000 7f778f183980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.851+0000 7f778f183980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.851+0000 7f778f183980 20 realm 2024-03-04T17:45:20.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.851+0000 7f778f183980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.851+0000 7f778f183980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.851+0000 7f778f183980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:20.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.851+0000 7f778f183980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.851+0000 7f778f183980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:20.855 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.851+0000 7f778f183980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.855 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.855+0000 7f778f183980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:20.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.855+0000 7f778f183980 20 rados->read ofs=0 len=0 2024-03-04T17:45:20.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.855+0000 7f778f183980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:20.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.855+0000 7f778f183980 20 searching for the correct realm 2024-03-04T17:45:20.860 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:20.859+0000 7f778d49c640 0 -- 172.21.15.137:0/254045184 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f7784008050 msgr2=0x7f77840143f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:21.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.075+0000 7f778d49c640 0 -- 172.21.15.137:0/254045184 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55ae31a94210 msgr2=0x55ae31a966d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:21.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.083+0000 7f778f183980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:21.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.083+0000 7f778f183980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:21.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.083+0000 7f778f183980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:21.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.083+0000 7f778f183980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:21.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.083+0000 7f778f183980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:21.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.083+0000 7f778f183980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:21.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.083+0000 7f778f183980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:21.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:21.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 20 zone default found 2024-03-04T17:45:21.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 4 Realm: () 2024-03-04T17:45:21.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:21.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:21.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:21.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 20 zonegroup default 2024-03-04T17:45:21.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.087+0000 7f778f183980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.091 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.091+0000 7f778f183980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.091 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.091+0000 7f778f183980 20 started sync module instance, tier type = 2024-03-04T17:45:21.091 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.091+0000 7f778f183980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:21.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.115+0000 7f778f183980 20 add_watcher() i=0 2024-03-04T17:45:21.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.115+0000 7f778f183980 20 add_watcher() i=1 2024-03-04T17:45:21.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.115+0000 7f778f183980 20 add_watcher() i=2 2024-03-04T17:45:21.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.115+0000 7f778f183980 20 add_watcher() i=3 2024-03-04T17:45:21.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.115+0000 7f778f183980 20 add_watcher() i=4 2024-03-04T17:45:21.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.115+0000 7f778f183980 20 add_watcher() i=5 2024-03-04T17:45:21.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.115+0000 7f778f183980 20 add_watcher() i=6 2024-03-04T17:45:21.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.119+0000 7f778f183980 20 add_watcher() i=7 2024-03-04T17:45:21.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.119+0000 7f778f183980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:21.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.123+0000 7f7642fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:21.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.123+0000 7f778f183980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:21.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.123+0000 7f778f183980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:21.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.123+0000 7f778f183980 5 note: GC not initialized 2024-03-04T17:45:21.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.123+0000 7f7641fe3640 20 reqs_thread_entry: start 2024-03-04T17:45:21.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.123+0000 7f778f183980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:21.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.123+0000 7f778f183980 20 Filter name: none 2024-03-04T17:45:21.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.127+0000 7f75b37fe640 20 reqs_thread_entry: start 2024-03-04T17:45:21.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.127+0000 7f778f183980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.127 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.127+0000 7f778f183980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:21.128 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.127+0000 7f778f183980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.127+0000 7f778f183980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:21.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.127+0000 7f778f183980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:21.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.127+0000 7f778f183980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-03-04T17:45:21.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.127+0000 7f778f183980 20 INFO: main: starting shard=0 2024-03-04T17:45:21.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.127+0000 7f778f183980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:21.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.131+0000 7f778f183980 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, marker= 2024-03-04T17:45:21.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.131+0000 7f778f183980 20 INFO: main: done 2024-03-04T17:45:21.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.131+0000 7f7642fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:21.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.135+0000 7f778f183980 20 remove_watcher() i=0 2024-03-04T17:45:21.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.135+0000 7f778f183980 2 removed watcher, disabling cache 2024-03-04T17:45:21.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.139+0000 7f778f183980 20 remove_watcher() i=1 2024-03-04T17:45:21.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.139+0000 7f778f183980 20 remove_watcher() i=2 2024-03-04T17:45:21.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.143+0000 7f778f183980 20 remove_watcher() i=3 2024-03-04T17:45:21.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.147+0000 7f778f183980 20 remove_watcher() i=4 2024-03-04T17:45:21.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.147+0000 7f778f183980 20 remove_watcher() i=5 2024-03-04T17:45:21.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.151+0000 7f778f183980 20 remove_watcher() i=6 2024-03-04T17:45:21.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.155+0000 7f778f183980 20 remove_watcher() i=7 2024-03-04T17:45:21.171 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-03-04T17:45:21.204 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:21.204 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:21.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.243+0000 7f3a4901e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.247 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.247+0000 7f3a4901e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.247 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.247+0000 7f3a4901e980 20 realm 2024-03-04T17:45:21.247 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.247+0000 7f3a4901e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.247 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.247+0000 7f3a4901e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.247 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.247+0000 7f3a4901e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:21.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.247+0000 7f3a4901e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.247+0000 7f3a4901e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.247+0000 7f3a4901e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.247+0000 7f3a4901e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:21.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.247+0000 7f3a4901e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.247+0000 7f3a4901e980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:21.250 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.247+0000 7f3a4901e980 20 searching for the correct realm 2024-03-04T17:45:21.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.255+0000 7f3a47337640 0 -- 172.21.15.137:0/1609188447 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f3a400cc430 msgr2=0x7f3a400cc890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:21.472 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.471+0000 7f3a47337640 0 -- 172.21.15.137:0/1609188447 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x5568e90372a0 msgr2=0x5568e9039760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:21.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.479+0000 7f3a4901e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:21.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.479+0000 7f3a4901e980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:21.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.479+0000 7f3a4901e980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:21.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.479+0000 7f3a4901e980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:21.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.479+0000 7f3a4901e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:21.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.479+0000 7f3a4901e980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:21.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.479+0000 7f3a4901e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.479+0000 7f3a4901e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.479+0000 7f3a4901e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.479+0000 7f3a4901e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:21.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.479+0000 7f3a4901e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:21.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 20 zone default found 2024-03-04T17:45:21.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 4 Realm: () 2024-03-04T17:45:21.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:21.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:21.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:21.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 20 zonegroup default 2024-03-04T17:45:21.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 20 started sync module instance, tier type = 2024-03-04T17:45:21.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.483+0000 7f3a4901e980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:21.509 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.507+0000 7f3a4901e980 20 add_watcher() i=0 2024-03-04T17:45:21.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.507+0000 7f3a4901e980 20 add_watcher() i=1 2024-03-04T17:45:21.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.507+0000 7f3a4901e980 20 add_watcher() i=2 2024-03-04T17:45:21.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.507+0000 7f3a4901e980 20 add_watcher() i=3 2024-03-04T17:45:21.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.507+0000 7f3a4901e980 20 add_watcher() i=4 2024-03-04T17:45:21.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.507+0000 7f3a4901e980 20 add_watcher() i=5 2024-03-04T17:45:21.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.507+0000 7f3a4901e980 20 add_watcher() i=6 2024-03-04T17:45:21.513 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.511+0000 7f3a4901e980 20 add_watcher() i=7 2024-03-04T17:45:21.513 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.511+0000 7f3a4901e980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:21.517 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.515+0000 7f38fd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:21.518 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3a4901e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:21.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3a4901e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:21.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3a4901e980 5 note: GC not initialized 2024-03-04T17:45:21.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3877fff640 20 reqs_thread_entry: start 2024-03-04T17:45:21.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3a4901e980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:21.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3a4901e980 20 Filter name: none 2024-03-04T17:45:21.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3875ffb640 20 reqs_thread_entry: start 2024-03-04T17:45:21.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3a4901e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3a4901e980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:21.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3a4901e980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3a4901e980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:21.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3a4901e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:21.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3a4901e980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-03-04T17:45:21.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3a4901e980 20 INFO: main: starting shard=0 2024-03-04T17:45:21.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.519+0000 7f3a4901e980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:21.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.523+0000 7f3a4901e980 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, marker= 2024-03-04T17:45:21.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.523+0000 7f3a4901e980 20 INFO: main: done 2024-03-04T17:45:21.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.523+0000 7f38fd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:21.530 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.527+0000 7f3a4901e980 20 remove_watcher() i=0 2024-03-04T17:45:21.530 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.527+0000 7f3a4901e980 2 removed watcher, disabling cache 2024-03-04T17:45:21.532 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.531+0000 7f3a4901e980 20 remove_watcher() i=1 2024-03-04T17:45:21.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.531+0000 7f3a4901e980 20 remove_watcher() i=2 2024-03-04T17:45:21.537 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.535+0000 7f3a4901e980 20 remove_watcher() i=3 2024-03-04T17:45:21.539 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.539+0000 7f3a4901e980 20 remove_watcher() i=4 2024-03-04T17:45:21.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.539+0000 7f3a4901e980 20 remove_watcher() i=5 2024-03-04T17:45:21.544 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.543+0000 7f3a4901e980 20 remove_watcher() i=6 2024-03-04T17:45:21.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.547+0000 7f3a4901e980 20 remove_watcher() i=7 2024-03-04T17:45:21.562 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: bucket check olh finds leftover OLH entries 2024-03-04T17:45:21.562 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:45:21.562 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2024-03-04T17:45:21.591 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:21.591 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:21.633 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.631+0000 7f96701fc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.635+0000 7f96701fc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.635+0000 7f96701fc980 20 realm 2024-03-04T17:45:21.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.635+0000 7f96701fc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.635 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.635+0000 7f96701fc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.635 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.635+0000 7f96701fc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:21.635 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.635+0000 7f96701fc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.635 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.635+0000 7f96701fc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.635 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.635+0000 7f96701fc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.636 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.635+0000 7f96701fc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:21.636 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.635+0000 7f96701fc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.637 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.635+0000 7f96701fc980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:21.637 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.635+0000 7f96701fc980 20 searching for the correct realm 2024-03-04T17:45:21.644 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.643+0000 7f966e515640 0 -- 172.21.15.137:0/2260906641 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f966803d910 msgr2=0x7f9668018410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:21.860 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.859+0000 7f966e515640 0 -- 172.21.15.137:0/2260906641 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55e7cc16cf40 msgr2=0x55e7cc16f400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:21.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.867+0000 7f96701fc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:21.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.867+0000 7f96701fc980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:21.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.867+0000 7f96701fc980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:21.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.867+0000 7f96701fc980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:21.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.867+0000 7f96701fc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:21.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.867+0000 7f96701fc980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:21.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.867+0000 7f96701fc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.867+0000 7f96701fc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.867+0000 7f96701fc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.871 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:21.871 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.871 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:21.871 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 20 zone default found 2024-03-04T17:45:21.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 4 Realm: () 2024-03-04T17:45:21.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:21.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:21.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:21.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 20 zonegroup default 2024-03-04T17:45:21.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.874 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:21.874 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 20 started sync module instance, tier type = 2024-03-04T17:45:21.874 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.871+0000 7f96701fc980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:21.899 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.899+0000 7f96701fc980 20 add_watcher() i=0 2024-03-04T17:45:21.899 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.899+0000 7f96701fc980 20 add_watcher() i=1 2024-03-04T17:45:21.900 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.899+0000 7f96701fc980 20 add_watcher() i=2 2024-03-04T17:45:21.900 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.899+0000 7f96701fc980 20 add_watcher() i=3 2024-03-04T17:45:21.900 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.899+0000 7f96701fc980 20 add_watcher() i=4 2024-03-04T17:45:21.900 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.899+0000 7f96701fc980 20 add_watcher() i=5 2024-03-04T17:45:21.900 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.899+0000 7f96701fc980 20 add_watcher() i=6 2024-03-04T17:45:21.902 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.899+0000 7f96701fc980 20 add_watcher() i=7 2024-03-04T17:45:21.902 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.899+0000 7f96701fc980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:21.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f951f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:21.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f96701fc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:21.908 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f96701fc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:21.908 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f96701fc980 5 note: GC not initialized 2024-03-04T17:45:21.908 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f951e7fc640 20 reqs_thread_entry: start 2024-03-04T17:45:21.908 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f96701fc980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:21.908 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f96701fc980 20 Filter name: none 2024-03-04T17:45:21.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f948ffff640 20 reqs_thread_entry: start 2024-03-04T17:45:21.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f96701fc980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:21.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f96701fc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f96701fc980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:21.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f96701fc980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:21.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f96701fc980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:21.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f96701fc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 : miss 2024-03-04T17:45:21.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.907+0000 7f96701fc980 20 rados->read ofs=0 len=0 2024-03-04T17:45:21.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.911+0000 7f96701fc980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:21.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.911+0000 7f96701fc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 info.flags=0x17 2024-03-04T17:45:21.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.911+0000 7f96701fc980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 to cache LRU end 2024-03-04T17:45:21.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.911+0000 7f96701fc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:21.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.911+0000 7f96701fc980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:21.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.911+0000 7f96701fc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 2024-03-04T17:45:21.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.911+0000 7f96701fc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:21.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.911+0000 7f96701fc980 1 NOTICE: finished shard 0 (2 entries found) 2024-03-04T17:45:21.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.911+0000 7f96701fc980 1 NOTICE: finished all shards (2 entries found) 2024-03-04T17:45:21.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.915+0000 7f951f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:21.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.919+0000 7f96701fc980 20 remove_watcher() i=0 2024-03-04T17:45:21.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.919+0000 7f96701fc980 2 removed watcher, disabling cache 2024-03-04T17:45:21.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.919+0000 7f96701fc980 20 remove_watcher() i=1 2024-03-04T17:45:21.924 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.923+0000 7f96701fc980 20 remove_watcher() i=2 2024-03-04T17:45:21.926 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.927+0000 7f96701fc980 20 remove_watcher() i=3 2024-03-04T17:45:21.928 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.927+0000 7f96701fc980 20 remove_watcher() i=4 2024-03-04T17:45:21.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.931+0000 7f96701fc980 20 remove_watcher() i=5 2024-03-04T17:45:21.934 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.931+0000 7f96701fc980 20 remove_watcher() i=6 2024-03-04T17:45:21.937 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:21.935+0000 7f96701fc980 20 remove_watcher() i=7 2024-03-04T17:45:21.959 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: bucket check olh fixes leftover OLH entries 2024-03-04T17:45:21.959 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:45:21.960 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket check olh --bucket check-bucket --fix --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-03-04T17:45:21.991 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:21.991 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:22.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.035+0000 7f5eb7674980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.035+0000 7f5eb7674980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.035+0000 7f5eb7674980 20 realm 2024-03-04T17:45:22.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.035+0000 7f5eb7674980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.035+0000 7f5eb7674980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.035+0000 7f5eb7674980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:22.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.035+0000 7f5eb7674980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.039+0000 7f5eb7674980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.039+0000 7f5eb7674980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.244 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.243+0000 7f5eb7674980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:22.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.243+0000 7f5eb7674980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.247+0000 7f5eb7674980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:22.247 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.247+0000 7f5eb7674980 20 searching for the correct realm 2024-03-04T17:45:22.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.251+0000 7f5eb498b640 0 -- 172.21.15.137:0/2807433883 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f5ea8008040 msgr2=0x7f5ea803f440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:22.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.463+0000 7f5eb498b640 0 -- 172.21.15.137:0/2807433883 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x560a4c8dd620 msgr2=0x560a4c8dfa10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:22.480 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.479+0000 7f5eb7674980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:22.480 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.479+0000 7f5eb7674980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:22.480 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.479+0000 7f5eb7674980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:22.480 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.479+0000 7f5eb7674980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:22.480 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.479+0000 7f5eb7674980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:22.480 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.479+0000 7f5eb7674980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:22.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.479+0000 7f5eb7674980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.479+0000 7f5eb7674980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.479+0000 7f5eb7674980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.479+0000 7f5eb7674980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:22.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.479+0000 7f5eb7674980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:22.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 20 zone default found 2024-03-04T17:45:22.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 4 Realm: () 2024-03-04T17:45:22.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:22.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:22.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:22.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 20 zonegroup default 2024-03-04T17:45:22.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 20 started sync module instance, tier type = 2024-03-04T17:45:22.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.483+0000 7f5eb7674980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:22.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.511+0000 7f5eb7674980 20 add_watcher() i=0 2024-03-04T17:45:22.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.511+0000 7f5eb7674980 20 add_watcher() i=1 2024-03-04T17:45:22.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.511+0000 7f5eb7674980 20 add_watcher() i=2 2024-03-04T17:45:22.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.511+0000 7f5eb7674980 20 add_watcher() i=3 2024-03-04T17:45:22.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.511+0000 7f5eb7674980 20 add_watcher() i=4 2024-03-04T17:45:22.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.511+0000 7f5eb7674980 20 add_watcher() i=5 2024-03-04T17:45:22.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.511+0000 7f5eb7674980 20 add_watcher() i=6 2024-03-04T17:45:22.513 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.511+0000 7f5eb7674980 20 add_watcher() i=7 2024-03-04T17:45:22.513 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.511+0000 7f5eb7674980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:22.518 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.515+0000 7f5d6b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:22.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.519+0000 7f5eb7674980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:22.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.519+0000 7f5eb7674980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:22.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.519+0000 7f5eb7674980 5 note: GC not initialized 2024-03-04T17:45:22.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.519+0000 7f5d6a7e4640 20 reqs_thread_entry: start 2024-03-04T17:45:22.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.519+0000 7f5eb7674980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:22.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.519+0000 7f5eb7674980 20 Filter name: none 2024-03-04T17:45:22.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.519+0000 7f5cdbfff640 20 reqs_thread_entry: start 2024-03-04T17:45:22.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.519+0000 7f5eb7674980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:22.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.519+0000 7f5eb7674980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.519+0000 7f5eb7674980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:22.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.519+0000 7f5eb7674980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:22.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.519+0000 7f5eb7674980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:22.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.519+0000 7f5eb7674980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 : miss 2024-03-04T17:45:22.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.519+0000 7f5eb7674980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.523+0000 7f5eb7674980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:22.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.523+0000 7f5eb7674980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 info.flags=0x17 2024-03-04T17:45:22.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.523+0000 7f5eb7674980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 to cache LRU end 2024-03-04T17:45:22.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.523+0000 7f5eb7674980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:22.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.523+0000 7f5eb7674980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:22.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.523+0000 7f5eb7674980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 2024-03-04T17:45:22.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.523+0000 7f5eb7674980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:22.524 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.523+0000 7f5eb7674980 20 get_obj_state: octx=0x560a4c667300 obj=check-bucket:c state=0x560a4c914218 s->prefetch_data=0 2024-03-04T17:45:22.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.523+0000 7f5eb7674980 20 get_obj_state: setting s->obj_tag to ygkw4jg0cs9h6691yi9ghvw830ty70xg 2024-03-04T17:45:22.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.523+0000 7f5eb7674980 20 get_obj_state_impl: setting s->olh_tag to ixksrzjdt0pwg5sfjlxsvb2gx4rtzjij 2024-03-04T17:45:22.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.523+0000 7f5eb7674980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:22.526 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.527+0000 7f5eb7674980 20 olh_log_entry: epoch=4 op=2 key=c[] 2024-03-04T17:45:22.526 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.527+0000 7f5eb7674980 20 olh_log_entry: epoch=4 op=3 key=c[EObd.Gdu4GYAts6x6tAlng6wq00OvHk] 2024-03-04T17:45:22.527 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.527+0000 7f5eb7674980 20 get_obj_state: octx=0x560a4c9412b0 obj=check-bucket:_:EObd.Gdu4GYAts6x6tAlng6wq00OvHk_c state=0x560a4c9133a8 s->prefetch_data=0 2024-03-04T17:45:22.528 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.527+0000 7f5eb7674980 20 get_obj_state: octx=0x560a4c9412b0 obj=check-bucket:c state=0x560a4c9135e8 s->prefetch_data=0 2024-03-04T17:45:22.529 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.527+0000 7f5eb7674980 20 get_obj_state: setting s->obj_tag to ygkw4jg0cs9h6691yi9ghvw830ty70xg 2024-03-04T17:45:22.529 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.527+0000 7f5eb7674980 20 get_obj_state_impl: setting s->olh_tag to ixksrzjdt0pwg5sfjlxsvb2gx4rtzjij 2024-03-04T17:45:22.529 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.527+0000 7f5eb7674980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=ixksrzjdt0pwg5sfjlxsvb2gx4rtzjij 2024-03-04T17:45:22.532 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.531+0000 7f5eb7674980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:22.536 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.535+0000 7f5eb7674980 20 get_obj_state: octx=0x560a4c667300 obj=check-bucket:d state=0x560a4c914218 s->prefetch_data=0 2024-03-04T17:45:22.537 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.535+0000 7f5eb7674980 20 get_obj_state: setting s->obj_tag to 9oxbb493sgz3f175eea8xx4zb2ogf397 2024-03-04T17:45:22.537 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.535+0000 7f5eb7674980 20 get_obj_state_impl: setting s->olh_tag to fvoe12fhqv9p23a3c58sza5pfp2o2jne 2024-03-04T17:45:22.537 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.535+0000 7f5eb7674980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:22.538 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.539+0000 7f5eb7674980 20 olh_log_entry: epoch=4 op=2 key=d[] 2024-03-04T17:45:22.538 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.539+0000 7f5eb7674980 20 olh_log_entry: epoch=4 op=3 key=d[XySj6PWa4sEIU95dJ9.m0WlQwNLtONU] 2024-03-04T17:45:22.538 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.539+0000 7f5eb7674980 20 get_obj_state: octx=0x560a4c9412b0 obj=check-bucket:_:XySj6PWa4sEIU95dJ9.m0WlQwNLtONU_d state=0x560a4c9167f8 s->prefetch_data=0 2024-03-04T17:45:22.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.543+0000 7f5eb7674980 20 get_obj_state: octx=0x560a4c9412b0 obj=check-bucket:d state=0x560a4c915798 s->prefetch_data=0 2024-03-04T17:45:22.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.543+0000 7f5eb7674980 20 get_obj_state: setting s->obj_tag to 9oxbb493sgz3f175eea8xx4zb2ogf397 2024-03-04T17:45:22.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.543+0000 7f5eb7674980 20 get_obj_state_impl: setting s->olh_tag to fvoe12fhqv9p23a3c58sza5pfp2o2jne 2024-03-04T17:45:22.544 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.543+0000 7f5eb7674980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=fvoe12fhqv9p23a3c58sza5pfp2o2jne 2024-03-04T17:45:22.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.547+0000 7f5eb7674980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:22.553 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.551+0000 7f5eb7674980 1 NOTICE: finished shard 0 (2 entries removed) 2024-03-04T17:45:22.553 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.551+0000 7f5eb7674980 1 NOTICE: finished all shards (2 entries removed) 2024-03-04T17:45:22.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.555+0000 7f5d6b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:22.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.559+0000 7f5eb7674980 20 remove_watcher() i=0 2024-03-04T17:45:22.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.559+0000 7f5eb7674980 2 removed watcher, disabling cache 2024-03-04T17:45:22.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.563+0000 7f5eb7674980 20 remove_watcher() i=1 2024-03-04T17:45:22.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.563+0000 7f5eb7674980 20 remove_watcher() i=2 2024-03-04T17:45:22.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.567+0000 7f5eb7674980 20 remove_watcher() i=3 2024-03-04T17:45:22.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.567+0000 7f5eb7674980 20 remove_watcher() i=4 2024-03-04T17:45:22.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.571+0000 7f5eb7674980 20 remove_watcher() i=5 2024-03-04T17:45:22.574 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.575+0000 7f5eb7674980 20 remove_watcher() i=6 2024-03-04T17:45:22.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.579+0000 7f5eb7674980 20 remove_watcher() i=7 2024-03-04T17:45:22.596 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-03-04T17:45:22.625 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:22.625 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:22.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.667+0000 7fb224a99980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.667+0000 7fb224a99980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.667+0000 7fb224a99980 20 realm 2024-03-04T17:45:22.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.667+0000 7fb224a99980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.667+0000 7fb224a99980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.667+0000 7fb224a99980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:22.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.667+0000 7fb224a99980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.671+0000 7fb224a99980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.671+0000 7fb224a99980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.671+0000 7fb224a99980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:22.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.671+0000 7fb224a99980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.672 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.671+0000 7fb224a99980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:22.672 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.671+0000 7fb224a99980 20 searching for the correct realm 2024-03-04T17:45:22.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.675+0000 7fb222db2640 0 -- 172.21.15.137:0/822546077 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fb21c0db910 msgr2=0x7fb21c0bf3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:22.892 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.891+0000 7fb222db2640 0 -- 172.21.15.137:0/822546077 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x556d09af0210 msgr2=0x556d09af26d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:22.900 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.899+0000 7fb224a99980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:22.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.899+0000 7fb224a99980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:22.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.899+0000 7fb224a99980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:22.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.899+0000 7fb224a99980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:22.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.899+0000 7fb224a99980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:22.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.899+0000 7fb224a99980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:22.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.899+0000 7fb224a99980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.899+0000 7fb224a99980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.899+0000 7fb224a99980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.902 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:22.902 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:22.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 20 zone default found 2024-03-04T17:45:22.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 4 Realm: () 2024-03-04T17:45:22.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:22.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:22.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:22.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 20 zonegroup default 2024-03-04T17:45:22.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.904 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.904 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:22.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 20 started sync module instance, tier type = 2024-03-04T17:45:22.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.903+0000 7fb224a99980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.931+0000 7fb224a99980 20 add_watcher() i=0 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.931+0000 7fb224a99980 20 add_watcher() i=1 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.931+0000 7fb224a99980 20 add_watcher() i=2 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.931+0000 7fb224a99980 20 add_watcher() i=3 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.931+0000 7fb224a99980 20 add_watcher() i=4 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.931+0000 7fb224a99980 20 add_watcher() i=5 2024-03-04T17:45:22.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.931+0000 7fb224a99980 20 add_watcher() i=6 2024-03-04T17:45:22.933 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.931+0000 7fb224a99980 20 add_watcher() i=7 2024-03-04T17:45:22.934 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.931+0000 7fb224a99980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:22.938 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.935+0000 7fb0d8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:22.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.939+0000 7fb224a99980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:22.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.939+0000 7fb224a99980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:22.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.939+0000 7fb224a99980 5 note: GC not initialized 2024-03-04T17:45:22.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.939+0000 7fb0537fe640 20 reqs_thread_entry: start 2024-03-04T17:45:22.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.939+0000 7fb224a99980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:22.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.939+0000 7fb224a99980 20 Filter name: none 2024-03-04T17:45:22.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.939+0000 7fb0517fa640 20 reqs_thread_entry: start 2024-03-04T17:45:22.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.939+0000 7fb224a99980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.939+0000 7fb224a99980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:22.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.939+0000 7fb224a99980 20 rados->read ofs=0 len=0 2024-03-04T17:45:22.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.943+0000 7fb224a99980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:22.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.943+0000 7fb224a99980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:22.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.943+0000 7fb224a99980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-03-04T17:45:22.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.943+0000 7fb224a99980 20 INFO: main: starting shard=0 2024-03-04T17:45:22.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.943+0000 7fb224a99980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:22.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.943+0000 7fb224a99980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:22.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.943+0000 7fb224a99980 20 INFO: main: done 2024-03-04T17:45:22.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.943+0000 7fb0d8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:22.950 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.951+0000 7fb224a99980 20 remove_watcher() i=0 2024-03-04T17:45:22.950 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.951+0000 7fb224a99980 2 removed watcher, disabling cache 2024-03-04T17:45:22.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.951+0000 7fb224a99980 20 remove_watcher() i=1 2024-03-04T17:45:22.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.955+0000 7fb224a99980 20 remove_watcher() i=2 2024-03-04T17:45:22.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.955+0000 7fb224a99980 20 remove_watcher() i=3 2024-03-04T17:45:22.959 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.959+0000 7fb224a99980 20 remove_watcher() i=4 2024-03-04T17:45:22.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.959+0000 7fb224a99980 20 remove_watcher() i=5 2024-03-04T17:45:22.965 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.963+0000 7fb224a99980 20 remove_watcher() i=6 2024-03-04T17:45:22.967 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:22.967+0000 7fb224a99980 20 remove_watcher() i=7 2024-03-04T17:45:22.983 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-03-04T17:45:23.012 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:23.012 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:23.054 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.051+0000 7f81ada12980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.056 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.055+0000 7f81ada12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.056 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.055+0000 7f81ada12980 20 realm 2024-03-04T17:45:23.056 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.055+0000 7f81ada12980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.055+0000 7f81ada12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.055+0000 7f81ada12980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:23.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.055+0000 7f81ada12980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.055+0000 7f81ada12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.055+0000 7f81ada12980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.059+0000 7f81ada12980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:23.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.059+0000 7f81ada12980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.059 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.059+0000 7f81ada12980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:23.059 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.059+0000 7f81ada12980 20 searching for the correct realm 2024-03-04T17:45:23.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.063+0000 7f81abd2b640 0 -- 172.21.15.137:0/40030196 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f81a407fb90 msgr2=0x7f81a405c0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:23.280 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.279+0000 7f81abd2b640 0 -- 172.21.15.137:0/40030196 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x558aeb563390 msgr2=0x558aeb565850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:23.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.287+0000 7f81ada12980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:23.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.287+0000 7f81ada12980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:23.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.287+0000 7f81ada12980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:23.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.287+0000 7f81ada12980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:23.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.287+0000 7f81ada12980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:23.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.287+0000 7f81ada12980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:23.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.287+0000 7f81ada12980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.291 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.291+0000 7f81ada12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.291 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.291+0000 7f81ada12980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.292 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.291+0000 7f81ada12980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:23.292 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.291+0000 7f81ada12980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.291+0000 7f81ada12980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:23.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.291+0000 7f81ada12980 20 zone default found 2024-03-04T17:45:23.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.291+0000 7f81ada12980 4 Realm: () 2024-03-04T17:45:23.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.291+0000 7f81ada12980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:23.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.291+0000 7f81ada12980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:23.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.291+0000 7f81ada12980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:23.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.291+0000 7f81ada12980 20 zonegroup default 2024-03-04T17:45:23.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.291+0000 7f81ada12980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.291+0000 7f81ada12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.291+0000 7f81ada12980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.295+0000 7f81ada12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.295+0000 7f81ada12980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.295+0000 7f81ada12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.295+0000 7f81ada12980 20 started sync module instance, tier type = 2024-03-04T17:45:23.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.295+0000 7f81ada12980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:23.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.319+0000 7f81ada12980 20 add_watcher() i=0 2024-03-04T17:45:23.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.319+0000 7f81ada12980 20 add_watcher() i=1 2024-03-04T17:45:23.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.319+0000 7f81ada12980 20 add_watcher() i=2 2024-03-04T17:45:23.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.319+0000 7f81ada12980 20 add_watcher() i=3 2024-03-04T17:45:23.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.319+0000 7f81ada12980 20 add_watcher() i=4 2024-03-04T17:45:23.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.319+0000 7f81ada12980 20 add_watcher() i=5 2024-03-04T17:45:23.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.319+0000 7f81ada12980 20 add_watcher() i=6 2024-03-04T17:45:23.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.319+0000 7f81ada12980 20 add_watcher() i=7 2024-03-04T17:45:23.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.319+0000 7f81ada12980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:23.326 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.323+0000 7f8061fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:23.327 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f81ada12980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:23.327 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f81ada12980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:23.327 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f81ada12980 5 note: GC not initialized 2024-03-04T17:45:23.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f8060fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:23.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f81ada12980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:23.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f81ada12980 20 Filter name: none 2024-03-04T17:45:23.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f7fda7fc640 20 reqs_thread_entry: start 2024-03-04T17:45:23.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f81ada12980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f81ada12980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:23.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f81ada12980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f81ada12980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:23.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f81ada12980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:23.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f81ada12980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-03-04T17:45:23.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f81ada12980 20 INFO: main: starting shard=0 2024-03-04T17:45:23.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.327+0000 7f81ada12980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:23.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.331+0000 7f81ada12980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T17:45:23.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.331+0000 7f81ada12980 20 INFO: main: done 2024-03-04T17:45:23.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.331+0000 7f8061fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:23.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.339+0000 7f81ada12980 20 remove_watcher() i=0 2024-03-04T17:45:23.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.339+0000 7f81ada12980 2 removed watcher, disabling cache 2024-03-04T17:45:23.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.339+0000 7f81ada12980 20 remove_watcher() i=1 2024-03-04T17:45:23.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.343+0000 7f81ada12980 20 remove_watcher() i=2 2024-03-04T17:45:23.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.343+0000 7f81ada12980 20 remove_watcher() i=3 2024-03-04T17:45:23.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.347+0000 7f81ada12980 20 remove_watcher() i=4 2024-03-04T17:45:23.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.347+0000 7f81ada12980 20 remove_watcher() i=5 2024-03-04T17:45:23.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.351+0000 7f81ada12980 20 remove_watcher() i=6 2024-03-04T17:45:23.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.355+0000 7f81ada12980 20 remove_watcher() i=7 2024-03-04T17:45:23.372 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: bucket check olh does not find new leftover OLH entries after fix 2024-03-04T17:45:23.372 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:45:23.372 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2024-03-04T17:45:23.402 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:23.402 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:23.444 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.443+0000 7fe9be64d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.447+0000 7fe9be64d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.447+0000 7fe9be64d980 20 realm 2024-03-04T17:45:23.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.447+0000 7fe9be64d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.447+0000 7fe9be64d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.447+0000 7fe9be64d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:23.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.447+0000 7fe9be64d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.447+0000 7fe9be64d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.447+0000 7fe9be64d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.448 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.447+0000 7fe9be64d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:23.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.447+0000 7fe9be64d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.447+0000 7fe9be64d980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:23.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.447+0000 7fe9be64d980 20 searching for the correct realm 2024-03-04T17:45:23.454 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.455+0000 7fe9bc966640 0 -- 172.21.15.137:0/99433654 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fe9b407ebf0 msgr2=0x7fe9b4071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:23.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.671+0000 7fe9bc966640 0 -- 172.21.15.137:0/99433654 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x560cf00b5e10 msgr2=0x560cf00b82d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:23.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.679+0000 7fe9be64d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:23.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.679+0000 7fe9be64d980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:23.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.679+0000 7fe9be64d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:23.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.679+0000 7fe9be64d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:23.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.679+0000 7fe9be64d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:23.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.679+0000 7fe9be64d980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:23.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.679+0000 7fe9be64d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.679+0000 7fe9be64d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.679+0000 7fe9be64d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.679+0000 7fe9be64d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:23.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.679+0000 7fe9be64d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:23.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 20 zone default found 2024-03-04T17:45:23.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 4 Realm: () 2024-03-04T17:45:23.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:23.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:23.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:23.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 20 zonegroup default 2024-03-04T17:45:23.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:23.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 20 started sync module instance, tier type = 2024-03-04T17:45:23.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.683+0000 7fe9be64d980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:23.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.707+0000 7fe9be64d980 20 add_watcher() i=0 2024-03-04T17:45:23.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.707+0000 7fe9be64d980 20 add_watcher() i=1 2024-03-04T17:45:23.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.707+0000 7fe9be64d980 20 add_watcher() i=2 2024-03-04T17:45:23.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.707+0000 7fe9be64d980 20 add_watcher() i=3 2024-03-04T17:45:23.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.707+0000 7fe9be64d980 20 add_watcher() i=4 2024-03-04T17:45:23.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.707+0000 7fe9be64d980 20 add_watcher() i=5 2024-03-04T17:45:23.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.707+0000 7fe9be64d980 20 add_watcher() i=6 2024-03-04T17:45:23.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.711+0000 7fe9be64d980 20 add_watcher() i=7 2024-03-04T17:45:23.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.711+0000 7fe9be64d980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:23.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.715+0000 7fe872fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:23.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.715+0000 7fe9be64d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:23.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.715+0000 7fe9be64d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:23.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.715+0000 7fe9be64d980 5 note: GC not initialized 2024-03-04T17:45:23.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.715+0000 7fe871fdb640 20 reqs_thread_entry: start 2024-03-04T17:45:23.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.715+0000 7fe9be64d980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:23.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.715+0000 7fe9be64d980 20 Filter name: none 2024-03-04T17:45:23.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.715+0000 7fe7e37fe640 20 reqs_thread_entry: start 2024-03-04T17:45:23.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.715+0000 7fe9be64d980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:23.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.715+0000 7fe9be64d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.719+0000 7fe9be64d980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:23.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.719+0000 7fe9be64d980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:23.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.719+0000 7fe9be64d980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:23.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.719+0000 7fe9be64d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 : miss 2024-03-04T17:45:23.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.719+0000 7fe9be64d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:23.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.719+0000 7fe9be64d980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:23.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.719+0000 7fe9be64d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 info.flags=0x17 2024-03-04T17:45:23.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.719+0000 7fe9be64d980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 to cache LRU end 2024-03-04T17:45:23.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.719+0000 7fe9be64d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:23.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.719+0000 7fe9be64d980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:23.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.719+0000 7fe9be64d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 2024-03-04T17:45:23.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.719+0000 7fe9be64d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:23.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.719+0000 7fe9be64d980 1 NOTICE: finished shard 0 (0 entries found) 2024-03-04T17:45:23.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.719+0000 7fe9be64d980 1 NOTICE: finished all shards (0 entries found) 2024-03-04T17:45:23.723 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.723+0000 7fe872fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:23.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.727+0000 7fe9be64d980 20 remove_watcher() i=0 2024-03-04T17:45:23.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.727+0000 7fe9be64d980 2 removed watcher, disabling cache 2024-03-04T17:45:23.729 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.727+0000 7fe9be64d980 20 remove_watcher() i=1 2024-03-04T17:45:23.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.731+0000 7fe9be64d980 20 remove_watcher() i=2 2024-03-04T17:45:23.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.731+0000 7fe9be64d980 20 remove_watcher() i=3 2024-03-04T17:45:23.736 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.735+0000 7fe9be64d980 20 remove_watcher() i=4 2024-03-04T17:45:23.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.739+0000 7fe9be64d980 20 remove_watcher() i=5 2024-03-04T17:45:23.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.739+0000 7fe9be64d980 20 remove_watcher() i=6 2024-03-04T17:45:23.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:23.743+0000 7fe9be64d980 20 remove_watcher() i=7 2024-03-04T17:45:23.763 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: verify that bucket check fixes do not affect null version objects 2024-03-04T17:45:23.763 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:45:23.765 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:23.765 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:23.791 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:23.791 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:23.821 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:23.821 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:23.926 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: bucket check stats are correct in the presence of unlinked entries 2024-03-04T17:45:23.926 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:45:23.928 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:23.928 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:24.030 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.030 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:24.573 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.573 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:24.714 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.714 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:24.780 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.780 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:24.908 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.908 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:24.972 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:24.973 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:25.104 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:25.105 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:25.169 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:25.169 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:25.294 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:25.294 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:25.357 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:25.357 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:25.485 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:25.486 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:25.549 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:25.549 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:25.673 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:25.673 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:25.733 INFO:tasks.workunit.client.0.smithi137.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-03-04T17:45:26.156 INFO:tasks.workunit.client.0.smithi137.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-03-04T17:45:27.555 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:27.555 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:27.691 INFO:tasks.workunit.client.0.smithi137.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:27.691 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-03-04T17:45:27.727 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:27.727 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:27.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.767+0000 7fcc25eb8980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.771+0000 7fcc25eb8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:27.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.771+0000 7fcc25eb8980 20 realm 2024-03-04T17:45:27.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.771+0000 7fcc25eb8980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.771+0000 7fcc25eb8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:27.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.771+0000 7fcc25eb8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:27.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.771+0000 7fcc25eb8980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.771+0000 7fcc25eb8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:27.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.771+0000 7fcc25eb8980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.771+0000 7fcc25eb8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:27.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.771+0000 7fcc25eb8980 20 rados->read ofs=0 len=0 2024-03-04T17:45:27.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.771+0000 7fcc25eb8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:27.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.771+0000 7fcc25eb8980 20 searching for the correct realm 2024-03-04T17:45:27.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.779+0000 7fcc241d1640 0 -- 172.21.15.137:0/1979448457 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fcc1c112170 msgr2=0x7fcc1c0d6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:27.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:27.995+0000 7fcc241d1640 0 -- 172.21.15.137:0/1979448457 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55c079d59f30 msgr2=0x55c079d5c3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:28.005 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.003+0000 7fcc25eb8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:28.005 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.003+0000 7fcc25eb8980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:28.005 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.003+0000 7fcc25eb8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:28.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.003+0000 7fcc25eb8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:28.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.003+0000 7fcc25eb8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:28.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.003+0000 7fcc25eb8980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:28.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.003+0000 7fcc25eb8980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.007 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:28.007 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:28.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 20 zone default found 2024-03-04T17:45:28.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 4 Realm: () 2024-03-04T17:45:28.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:28.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:28.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:28.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 20 zonegroup default 2024-03-04T17:45:28.009 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.009 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.009 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.010 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.010 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.007+0000 7fcc25eb8980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.011 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.011+0000 7fcc25eb8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.011 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.011+0000 7fcc25eb8980 20 started sync module instance, tier type = 2024-03-04T17:45:28.011 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.011+0000 7fcc25eb8980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:28.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.035+0000 7fcc25eb8980 20 add_watcher() i=0 2024-03-04T17:45:28.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.035+0000 7fcc25eb8980 20 add_watcher() i=1 2024-03-04T17:45:28.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.035+0000 7fcc25eb8980 20 add_watcher() i=2 2024-03-04T17:45:28.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.035+0000 7fcc25eb8980 20 add_watcher() i=3 2024-03-04T17:45:28.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.035+0000 7fcc25eb8980 20 add_watcher() i=4 2024-03-04T17:45:28.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.035+0000 7fcc25eb8980 20 add_watcher() i=5 2024-03-04T17:45:28.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.035+0000 7fcc25eb8980 20 add_watcher() i=6 2024-03-04T17:45:28.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.035+0000 7fcc25eb8980 20 add_watcher() i=7 2024-03-04T17:45:28.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.035+0000 7fcc25eb8980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:28.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.043+0000 7fcada7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:28.043 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.043+0000 7fcc25eb8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:28.044 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.043+0000 7fcc25eb8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:28.044 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.043+0000 7fcc25eb8980 5 note: GC not initialized 2024-03-04T17:45:28.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.043+0000 7fcad97e2640 20 reqs_thread_entry: start 2024-03-04T17:45:28.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.043+0000 7fcc25eb8980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:28.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.043+0000 7fcc25eb8980 20 Filter name: none 2024-03-04T17:45:28.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.043+0000 7fcad27fc640 20 reqs_thread_entry: start 2024-03-04T17:45:28.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.043+0000 7fcc25eb8980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.043+0000 7fcc25eb8980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:28.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.043+0000 7fcc25eb8980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.047+0000 7fcc25eb8980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:28.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.047+0000 7fcc25eb8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:28.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.047+0000 7fcc25eb8980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-03-04T17:45:28.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.047+0000 7fcc25eb8980 20 INFO: main: starting shard=0 2024-03-04T17:45:28.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.047+0000 7fcc25eb8980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:28.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.047+0000 7fcc25eb8980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-03-04T17:45:28.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.047+0000 7fcc25eb8980 20 INFO: main: done 2024-03-04T17:45:28.051 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.051+0000 7fcada7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:28.056 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.055+0000 7fcc25eb8980 20 remove_watcher() i=0 2024-03-04T17:45:28.056 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.055+0000 7fcc25eb8980 2 removed watcher, disabling cache 2024-03-04T17:45:28.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.055+0000 7fcc25eb8980 20 remove_watcher() i=1 2024-03-04T17:45:28.060 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.059+0000 7fcc25eb8980 20 remove_watcher() i=2 2024-03-04T17:45:28.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.063+0000 7fcc25eb8980 20 remove_watcher() i=3 2024-03-04T17:45:28.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.063+0000 7fcc25eb8980 20 remove_watcher() i=4 2024-03-04T17:45:28.066 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.067+0000 7fcc25eb8980 20 remove_watcher() i=5 2024-03-04T17:45:28.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.067+0000 7fcc25eb8980 20 remove_watcher() i=6 2024-03-04T17:45:28.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.071+0000 7fcc25eb8980 20 remove_watcher() i=7 2024-03-04T17:45:28.094 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:28.094 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:28.128 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:28.128 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:28.158 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:28.158 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:28.303 INFO:tasks.workunit.client.0.smithi137.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:28.304 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-03-04T17:45:28.337 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:28.338 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:28.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.379+0000 7fabdb2d3980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.383+0000 7fabdb2d3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.383+0000 7fabdb2d3980 20 realm 2024-03-04T17:45:28.384 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.383+0000 7fabdb2d3980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.384 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.383+0000 7fabdb2d3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.384 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.383+0000 7fabdb2d3980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:28.384 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.383+0000 7fabdb2d3980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.383+0000 7fabdb2d3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.383+0000 7fabdb2d3980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.383+0000 7fabdb2d3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:28.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.383+0000 7fabdb2d3980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.383+0000 7fabdb2d3980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:28.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.383+0000 7fabdb2d3980 20 searching for the correct realm 2024-03-04T17:45:28.392 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.391+0000 7fabd95ec640 0 -- 172.21.15.137:0/2803295575 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fabd003a040 msgr2=0x7fabd003a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:28.607 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.607+0000 7fabd95ec640 0 -- 172.21.15.137:0/2803295575 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x557d1ce65ee0 msgr2=0x557d1ce683a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:28.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:28.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:28.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:28.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:28.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:28.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:28.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.617 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.617 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.617 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:28.617 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:28.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 zone default found 2024-03-04T17:45:28.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 4 Realm: () 2024-03-04T17:45:28.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:28.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:28.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:28.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 zonegroup default 2024-03-04T17:45:28.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.615+0000 7fabdb2d3980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.619+0000 7fabdb2d3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.619+0000 7fabdb2d3980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.619+0000 7fabdb2d3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.619+0000 7fabdb2d3980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.620 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.619+0000 7fabdb2d3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.620 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.619+0000 7fabdb2d3980 20 started sync module instance, tier type = 2024-03-04T17:45:28.620 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.619+0000 7fabdb2d3980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:28.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.643+0000 7fabdb2d3980 20 add_watcher() i=0 2024-03-04T17:45:28.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.643+0000 7fabdb2d3980 20 add_watcher() i=1 2024-03-04T17:45:28.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.643+0000 7fabdb2d3980 20 add_watcher() i=2 2024-03-04T17:45:28.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.643+0000 7fabdb2d3980 20 add_watcher() i=3 2024-03-04T17:45:28.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.643+0000 7fabdb2d3980 20 add_watcher() i=4 2024-03-04T17:45:28.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.643+0000 7fabdb2d3980 20 add_watcher() i=5 2024-03-04T17:45:28.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.643+0000 7fabdb2d3980 20 add_watcher() i=6 2024-03-04T17:45:28.647 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.647+0000 7fabdb2d3980 20 add_watcher() i=7 2024-03-04T17:45:28.647 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.647+0000 7fabdb2d3980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:28.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.651+0000 7faa8efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:28.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.651+0000 7fabdb2d3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:28.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.651+0000 7fabdb2d3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:28.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.651+0000 7fabdb2d3980 5 note: GC not initialized 2024-03-04T17:45:28.653 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.651+0000 7faa8dfe3640 20 reqs_thread_entry: start 2024-03-04T17:45:28.653 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.651+0000 7fabdb2d3980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:28.653 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.651+0000 7fabdb2d3980 20 Filter name: none 2024-03-04T17:45:28.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.651+0000 7fa9ff7fe640 20 reqs_thread_entry: start 2024-03-04T17:45:28.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.651+0000 7fabdb2d3980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.655 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.655+0000 7fabdb2d3980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:28.655 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.655+0000 7fabdb2d3980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.655+0000 7fabdb2d3980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:28.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.655+0000 7fabdb2d3980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:28.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.655+0000 7fabdb2d3980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-03-04T17:45:28.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.655+0000 7fabdb2d3980 20 INFO: main: starting shard=0 2024-03-04T17:45:28.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.655+0000 7fabdb2d3980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:28.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.655+0000 7fabdb2d3980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-03-04T17:45:28.658 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.655+0000 7fabdb2d3980 20 INFO: main: done 2024-03-04T17:45:28.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.659+0000 7faa8efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:28.663 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.663+0000 7fabdb2d3980 20 remove_watcher() i=0 2024-03-04T17:45:28.664 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.663+0000 7fabdb2d3980 2 removed watcher, disabling cache 2024-03-04T17:45:28.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.663+0000 7fabdb2d3980 20 remove_watcher() i=1 2024-03-04T17:45:28.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.667+0000 7fabdb2d3980 20 remove_watcher() i=2 2024-03-04T17:45:28.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.667+0000 7fabdb2d3980 20 remove_watcher() i=3 2024-03-04T17:45:28.672 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.671+0000 7fabdb2d3980 20 remove_watcher() i=4 2024-03-04T17:45:28.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.675+0000 7fabdb2d3980 20 remove_watcher() i=5 2024-03-04T17:45:28.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.675+0000 7fabdb2d3980 20 remove_watcher() i=6 2024-03-04T17:45:28.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.679+0000 7fabdb2d3980 20 remove_watcher() i=7 2024-03-04T17:45:28.705 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:28.705 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:28.744 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:28.745 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:28.859 INFO:tasks.workunit.client.0.smithi137.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:28.860 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket check-bucket --object e 2024-03-04T17:45:28.894 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:28.954 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:28.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.935+0000 7fa16e10d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.939+0000 7fa16e10d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.939+0000 7fa16e10d980 20 realm 2024-03-04T17:45:28.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.939+0000 7fa16e10d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.939+0000 7fa16e10d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.939+0000 7fa16e10d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:28.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.939+0000 7fa16e10d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.939+0000 7fa16e10d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:28.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.939+0000 7fa16e10d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.939+0000 7fa16e10d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:28.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.939+0000 7fa16e10d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:28.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.943+0000 7fa16e10d980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:28.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.943+0000 7fa16e10d980 20 searching for the correct realm 2024-03-04T17:45:28.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:28.947+0000 7fa16c426640 0 -- 172.21.15.137:0/3042388219 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fa164061fd0 msgr2=0x7fa164058170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:29.163 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.163+0000 7fa16c426640 0 -- 172.21.15.137:0/3042388219 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x5588ac5ff240 msgr2=0x5588ac6016d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:29.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:29.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:29.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:29.173 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:29.173 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:29.173 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:29.173 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:29.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:29.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 zone default found 2024-03-04T17:45:29.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 4 Realm: () 2024-03-04T17:45:29.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:29.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:29.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:29.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 zonegroup default 2024-03-04T17:45:29.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.171+0000 7fa16e10d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.175+0000 7fa16e10d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.175+0000 7fa16e10d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.175+0000 7fa16e10d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.175+0000 7fa16e10d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.175+0000 7fa16e10d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.175+0000 7fa16e10d980 20 started sync module instance, tier type = 2024-03-04T17:45:29.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.175+0000 7fa16e10d980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:29.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.199+0000 7fa16e10d980 20 add_watcher() i=0 2024-03-04T17:45:29.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.199+0000 7fa16e10d980 20 add_watcher() i=1 2024-03-04T17:45:29.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.199+0000 7fa16e10d980 20 add_watcher() i=2 2024-03-04T17:45:29.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.199+0000 7fa16e10d980 20 add_watcher() i=3 2024-03-04T17:45:29.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.199+0000 7fa16e10d980 20 add_watcher() i=4 2024-03-04T17:45:29.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.199+0000 7fa16e10d980 20 add_watcher() i=5 2024-03-04T17:45:29.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.199+0000 7fa16e10d980 20 add_watcher() i=6 2024-03-04T17:45:29.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.199+0000 7fa16e10d980 20 add_watcher() i=7 2024-03-04T17:45:29.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.199+0000 7fa16e10d980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:29.206 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.203+0000 7fa021fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:29.207 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.207+0000 7fa16e10d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:29.207 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.207+0000 7fa16e10d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:29.207 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.207+0000 7fa16e10d980 5 note: GC not initialized 2024-03-04T17:45:29.208 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.207+0000 7fa020fe1640 20 reqs_thread_entry: start 2024-03-04T17:45:29.208 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.207+0000 7fa16e10d980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:29.208 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.207+0000 7fa16e10d980 20 Filter name: none 2024-03-04T17:45:29.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.207+0000 7f9f9a7fc640 20 reqs_thread_entry: start 2024-03-04T17:45:29.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.207+0000 7fa16e10d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.210 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.207+0000 7fa16e10d980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:29.210 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.207+0000 7fa16e10d980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.207+0000 7fa16e10d980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:29.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.207+0000 7fa16e10d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:29.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.207+0000 7fa16e10d980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-03-04T17:45:29.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.207+0000 7fa16e10d980 20 INFO: main: starting shard=0 2024-03-04T17:45:29.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.211+0000 7fa16e10d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:29.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.211+0000 7fa16e10d980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T17:45:29.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.211+0000 7fa16e10d980 20 INFO: main: done 2024-03-04T17:45:29.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.211+0000 7fa021fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:29.218 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.219+0000 7fa16e10d980 20 remove_watcher() i=0 2024-03-04T17:45:29.218 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.219+0000 7fa16e10d980 2 removed watcher, disabling cache 2024-03-04T17:45:29.220 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.219+0000 7fa16e10d980 20 remove_watcher() i=1 2024-03-04T17:45:29.222 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.219+0000 7fa16e10d980 20 remove_watcher() i=2 2024-03-04T17:45:29.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.223+0000 7fa16e10d980 20 remove_watcher() i=3 2024-03-04T17:45:29.226 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.227+0000 7fa16e10d980 20 remove_watcher() i=4 2024-03-04T17:45:29.229 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.227+0000 7fa16e10d980 20 remove_watcher() i=5 2024-03-04T17:45:29.232 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.231+0000 7fa16e10d980 20 remove_watcher() i=6 2024-03-04T17:45:29.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.235+0000 7fa16e10d980 20 remove_watcher() i=7 2024-03-04T17:45:29.252 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:29.252 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:29.280 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:29.280 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:29.383 INFO:tasks.workunit.client.0.smithi137.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T17:45:29.384 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket check-bucket --object f 2024-03-04T17:45:29.419 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:29.419 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:29.462 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.459+0000 7fbbdbb84980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.463+0000 7fbbdbb84980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.463+0000 7fbbdbb84980 20 realm 2024-03-04T17:45:29.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.463+0000 7fbbdbb84980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.463+0000 7fbbdbb84980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.463+0000 7fbbdbb84980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:29.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.463+0000 7fbbdbb84980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.463+0000 7fbbdbb84980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.463+0000 7fbbdbb84980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.467+0000 7fbbdbb84980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:29.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.467+0000 7fbbdbb84980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.467 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.467+0000 7fbbdbb84980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:29.467 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.467+0000 7fbbdbb84980 20 searching for the correct realm 2024-03-04T17:45:29.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.471+0000 7fbbd9e9d640 0 -- 172.21.15.137:0/171529453 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x555e01ca67b0 msgr2=0x555e01ca8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:29.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.675+0000 7fbbd8e9b640 0 -- 172.21.15.137:0/171529453 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7fbbcc042020 msgr2=0x7fbbcc08d130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:29.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.695+0000 7fbbdbb84980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:29.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.695+0000 7fbbdbb84980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:29.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.695+0000 7fbbdbb84980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:29.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.695+0000 7fbbdbb84980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:29.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.695+0000 7fbbdbb84980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:29.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.695+0000 7fbbdbb84980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:29.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.695+0000 7fbbdbb84980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.695+0000 7fbbdbb84980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.695+0000 7fbbdbb84980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.695+0000 7fbbdbb84980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:29.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.695+0000 7fbbdbb84980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:29.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 20 zone default found 2024-03-04T17:45:29.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 4 Realm: () 2024-03-04T17:45:29.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:29.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:29.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:29.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 20 zonegroup default 2024-03-04T17:45:29.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:29.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 20 started sync module instance, tier type = 2024-03-04T17:45:29.701 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.699+0000 7fbbdbb84980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:29.725 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.723+0000 7fbbdbb84980 20 add_watcher() i=0 2024-03-04T17:45:29.725 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.723+0000 7fbbdbb84980 20 add_watcher() i=1 2024-03-04T17:45:29.726 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.723+0000 7fbbdbb84980 20 add_watcher() i=2 2024-03-04T17:45:29.726 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.723+0000 7fbbdbb84980 20 add_watcher() i=3 2024-03-04T17:45:29.726 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.723+0000 7fbbdbb84980 20 add_watcher() i=4 2024-03-04T17:45:29.726 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.723+0000 7fbbdbb84980 20 add_watcher() i=5 2024-03-04T17:45:29.726 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.723+0000 7fbbdbb84980 20 add_watcher() i=6 2024-03-04T17:45:29.728 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.727+0000 7fbbdbb84980 20 add_watcher() i=7 2024-03-04T17:45:29.728 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.727+0000 7fbbdbb84980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:29.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.731+0000 7fba8ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:29.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.731+0000 7fbbdbb84980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:29.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.731+0000 7fbbdbb84980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:29.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.731+0000 7fbbdbb84980 5 note: GC not initialized 2024-03-04T17:45:29.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.731+0000 7fba8efe5640 20 reqs_thread_entry: start 2024-03-04T17:45:29.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.731+0000 7fbbdbb84980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:29.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.731+0000 7fbbdbb84980 20 Filter name: none 2024-03-04T17:45:29.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.735+0000 7fba8cfe1640 20 reqs_thread_entry: start 2024-03-04T17:45:29.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.735+0000 7fbbdbb84980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.735 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.735+0000 7fbbdbb84980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:29.735 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.735+0000 7fbbdbb84980 20 rados->read ofs=0 len=0 2024-03-04T17:45:29.737 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.735+0000 7fbbdbb84980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:29.737 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.735+0000 7fbbdbb84980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:29.737 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.735+0000 7fbbdbb84980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-03-04T17:45:29.737 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.735+0000 7fbbdbb84980 20 INFO: main: starting shard=0 2024-03-04T17:45:29.737 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.735+0000 7fbbdbb84980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:29.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.739+0000 7fbbdbb84980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T17:45:29.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.739+0000 7fbbdbb84980 20 INFO: main: done 2024-03-04T17:45:29.740 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.739+0000 7fba8ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:29.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.743+0000 7fbbdbb84980 20 remove_watcher() i=0 2024-03-04T17:45:29.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.743+0000 7fbbdbb84980 2 removed watcher, disabling cache 2024-03-04T17:45:29.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.747+0000 7fbbdbb84980 20 remove_watcher() i=1 2024-03-04T17:45:29.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.747+0000 7fbbdbb84980 20 remove_watcher() i=2 2024-03-04T17:45:29.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.751+0000 7fbbdbb84980 20 remove_watcher() i=3 2024-03-04T17:45:29.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.751+0000 7fbbdbb84980 20 remove_watcher() i=4 2024-03-04T17:45:29.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.755+0000 7fbbdbb84980 20 remove_watcher() i=5 2024-03-04T17:45:29.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.755+0000 7fbbdbb84980 20 remove_watcher() i=6 2024-03-04T17:45:29.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:29.759+0000 7fbbdbb84980 20 remove_watcher() i=7 2024-03-04T17:45:29.784 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:29.784 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:29.810 INFO:tasks.workunit.client.0.smithi137.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-03-04T17:45:30.224 INFO:tasks.workunit.client.0.smithi137.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-03-04T17:45:30.631 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2024-03-04T17:45:30.657 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:30.657 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:30.693 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.691+0000 7f198caef980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.695+0000 7f198caef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.695+0000 7f198caef980 20 realm 2024-03-04T17:45:30.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.695+0000 7f198caef980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.695+0000 7f198caef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.695+0000 7f198caef980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:30.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.695+0000 7f198caef980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.695+0000 7f198caef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.695+0000 7f198caef980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.695+0000 7f198caef980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:30.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.695+0000 7f198caef980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.699+0000 7f198caef980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:30.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.699+0000 7f198caef980 20 searching for the correct realm 2024-03-04T17:45:30.702 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.703+0000 7f1989e06640 0 -- 172.21.15.137:0/447210941 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f197c043310 msgr2=0x7f197c089270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:30.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.707+0000 7f1989e06640 0 -- 172.21.15.137:0/447210941 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x55b7006e4270 msgr2=0x55b7006e46b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:30.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:30.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:30.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:30.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:30.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:30.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:30.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:30.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:30.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 zone default found 2024-03-04T17:45:30.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 4 Realm: () 2024-03-04T17:45:30.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:30.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:30.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:30.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 zonegroup default 2024-03-04T17:45:30.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.715+0000 7f198caef980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.719+0000 7f198caef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.719+0000 7f198caef980 20 started sync module instance, tier type = 2024-03-04T17:45:30.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.719+0000 7f198caef980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:30.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.735+0000 7f198caef980 20 add_watcher() i=0 2024-03-04T17:45:30.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.735+0000 7f198caef980 20 add_watcher() i=1 2024-03-04T17:45:30.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.735+0000 7f198caef980 20 add_watcher() i=2 2024-03-04T17:45:30.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.735+0000 7f198caef980 20 add_watcher() i=3 2024-03-04T17:45:30.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.735+0000 7f198caef980 20 add_watcher() i=4 2024-03-04T17:45:30.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.735+0000 7f198caef980 20 add_watcher() i=5 2024-03-04T17:45:30.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.735+0000 7f198caef980 20 add_watcher() i=6 2024-03-04T17:45:30.740 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.739+0000 7f198caef980 20 add_watcher() i=7 2024-03-04T17:45:30.740 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.739+0000 7f198caef980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:30.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f1840fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:30.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:30.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:30.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 5 note: GC not initialized 2024-03-04T17:45:30.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f17bb7fe640 20 reqs_thread_entry: start 2024-03-04T17:45:30.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:30.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 20 Filter name: none 2024-03-04T17:45:30.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f17b97fa640 20 reqs_thread_entry: start 2024-03-04T17:45:30.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:30.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:30.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:30.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:30.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 : miss 2024-03-04T17:45:30.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:30.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 info.flags=0x17 2024-03-04T17:45:30.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 to cache LRU end 2024-03-04T17:45:30.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:30.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:30.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 2024-03-04T17:45:30.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 20 list_objects_ordered: starting attempt 1 2024-03-04T17:45:30.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.743+0000 7f198caef980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-03-04T17:45:30.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.747+0000 7f198caef980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T17:45:30.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.747+0000 7f198caef980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T17:45:30.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.747+0000 7f198caef980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T17:45:30.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.747+0000 7f198caef980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T17:45:30.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.755+0000 7f1840fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:30.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.759+0000 7f198caef980 20 remove_watcher() i=0 2024-03-04T17:45:30.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.759+0000 7f198caef980 2 removed watcher, disabling cache 2024-03-04T17:45:30.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.759+0000 7f198caef980 20 remove_watcher() i=1 2024-03-04T17:45:30.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.763+0000 7f198caef980 20 remove_watcher() i=2 2024-03-04T17:45:30.765 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.763+0000 7f198caef980 20 remove_watcher() i=3 2024-03-04T17:45:30.767 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.767+0000 7f198caef980 20 remove_watcher() i=4 2024-03-04T17:45:30.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.767+0000 7f198caef980 20 remove_watcher() i=5 2024-03-04T17:45:30.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.771+0000 7f198caef980 20 remove_watcher() i=6 2024-03-04T17:45:30.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.775+0000 7f198caef980 20 remove_watcher() i=7 2024-03-04T17:45:30.785 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-03-04T17:45:30.818 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:30.818 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:30.855 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.855+0000 7fed69462980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.857 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.855+0000 7fed69462980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.857 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.855+0000 7fed69462980 20 realm 2024-03-04T17:45:30.857 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.855+0000 7fed69462980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.855+0000 7fed69462980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.855+0000 7fed69462980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:30.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.855+0000 7fed69462980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.859+0000 7fed69462980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:30.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.859+0000 7fed69462980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.859 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.859+0000 7fed69462980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:30.859 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.859+0000 7fed69462980 20 rados->read ofs=0 len=0 2024-03-04T17:45:30.860 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.859+0000 7fed69462980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:30.860 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:30.859+0000 7fed69462980 20 searching for the correct realm 2024-03-04T17:45:31.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.071+0000 7fed6777b640 0 -- 172.21.15.137:0/1489945383 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fed6002b910 msgr2=0x7fed6000d3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:31.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.079+0000 7fed66f7a640 0 -- 172.21.15.137:0/1489945383 >> [v2:172.21.15.173:6805/879938785,v1:172.21.15.173:6808/879938785] conn(0x7fed58008050 msgr2=0x7fed58075430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:31.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:31.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:31.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:31.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:31.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:31.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:31.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:31.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:31.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 zone default found 2024-03-04T17:45:31.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 4 Realm: () 2024-03-04T17:45:31.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:31.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:31.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:31.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 zonegroup default 2024-03-04T17:45:31.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.091+0000 7fed69462980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.095+0000 7fed69462980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.095+0000 7fed69462980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.095+0000 7fed69462980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.095+0000 7fed69462980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.095+0000 7fed69462980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.095+0000 7fed69462980 20 started sync module instance, tier type = 2024-03-04T17:45:31.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.095+0000 7fed69462980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:31.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.119+0000 7fed69462980 20 add_watcher() i=0 2024-03-04T17:45:31.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.119+0000 7fed69462980 20 add_watcher() i=1 2024-03-04T17:45:31.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.119+0000 7fed69462980 20 add_watcher() i=2 2024-03-04T17:45:31.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.119+0000 7fed69462980 20 add_watcher() i=3 2024-03-04T17:45:31.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.119+0000 7fed69462980 20 add_watcher() i=4 2024-03-04T17:45:31.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.119+0000 7fed69462980 20 add_watcher() i=5 2024-03-04T17:45:31.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.119+0000 7fed69462980 20 add_watcher() i=6 2024-03-04T17:45:31.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.119+0000 7fed69462980 20 add_watcher() i=7 2024-03-04T17:45:31.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.119+0000 7fed69462980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:31.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7fec1d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:31.127 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7fed69462980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:31.127 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7fed69462980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:31.127 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7fed69462980 5 note: GC not initialized 2024-03-04T17:45:31.128 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7feb97fff640 20 reqs_thread_entry: start 2024-03-04T17:45:31.128 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7fed69462980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:31.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7fed69462980 20 Filter name: none 2024-03-04T17:45:31.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7feb95ffb640 20 reqs_thread_entry: start 2024-03-04T17:45:31.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7fed69462980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-03-04T17:45:31.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7fed69462980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7fed69462980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:31.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7fed69462980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-03-04T17:45:31.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7fed69462980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-03-04T17:45:31.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7fed69462980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 : miss 2024-03-04T17:45:31.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.127+0000 7fed69462980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.131+0000 7fed69462980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:31.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.131+0000 7fed69462980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 info.flags=0x17 2024-03-04T17:45:31.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.131+0000 7fed69462980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 to cache LRU end 2024-03-04T17:45:31.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.131+0000 7fed69462980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T17:45:31.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.131+0000 7fed69462980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-03-04T17:45:31.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.131+0000 7fed69462980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2 2024-03-04T17:45:31.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.131+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.134 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.135+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.135+0000 7fed69462980 20 get_obj_state: octx=0x555af3f97310 obj=check-bucket:c state=0x555af4243568 s->prefetch_data=0 2024-03-04T17:45:31.136 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.135+0000 7fed69462980 20 get_obj_state: setting s->obj_tag to gauz5bbr1145jqb35wngti03a07gwt4t 2024-03-04T17:45:31.136 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.135+0000 7fed69462980 20 get_obj_state_impl: setting s->olh_tag to 3t40fza6jk8ei33cjlumms16cq111b2g 2024-03-04T17:45:31.136 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.135+0000 7fed69462980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=3t40fza6jk8ei33cjlumms16cq111b2g 2024-03-04T17:45:31.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.135+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.141 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.139+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.143+0000 7fed69462980 20 olh_log_entry: epoch=3 op=3 key=c[osCVrHEaC9qGZvJFnOny3hOtFlUHrvo] 2024-03-04T17:45:31.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.143+0000 7fed69462980 20 get_obj_state: octx=0x555af3f97310 obj=check-bucket:_:osCVrHEaC9qGZvJFnOny3hOtFlUHrvo_c state=0x555af42426f8 s->prefetch_data=0 2024-03-04T17:45:31.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.143+0000 7fed69462980 10 manifest: total_size = 22 2024-03-04T17:45:31.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.143+0000 7fed69462980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17417121077300074039 2024-03-04T17:45:31.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.143+0000 7fed69462980 20 get_obj_state: octx=0x555af3f97310 obj=check-bucket:_:osCVrHEaC9qGZvJFnOny3hOtFlUHrvo_c state=0x555af42426f8 s->prefetch_data=0 2024-03-04T17:45:31.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.143+0000 7fed69462980 20 prepare_atomic_modification: state is not atomic. state=0x555af42426f8 2024-03-04T17:45:31.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.143+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.151+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.151+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.151+0000 7fec49ffb640 20 handle_completion(): completion ok for obj=c[osCVrHEaC9qGZvJFnOny3hOtFlUHrvo] 2024-03-04T17:45:31.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.155+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.155+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.159 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.159+0000 7fed69462980 20 get_obj_state: octx=0x555af3f97310 obj=check-bucket:d state=0x555af4243568 s->prefetch_data=0 2024-03-04T17:45:31.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.159+0000 7fed69462980 20 get_obj_state: setting s->obj_tag to 1dkpkjyhu7vd2y8581gxdj9zm34w5bf0 2024-03-04T17:45:31.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.159+0000 7fed69462980 20 get_obj_state_impl: setting s->olh_tag to z7lbykqz66w91zrmomxf657sozu1amkm 2024-03-04T17:45:31.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.159+0000 7fed69462980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=z7lbykqz66w91zrmomxf657sozu1amkm 2024-03-04T17:45:31.163 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.163+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.166 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.167+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.167+0000 7fed69462980 20 olh_log_entry: epoch=3 op=3 key=d[72vk4JYRc.nfRxnaogpSgQbDIQVNBLS] 2024-03-04T17:45:31.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.167+0000 7fed69462980 20 get_obj_state: octx=0x555af3f97310 obj=check-bucket:_:72vk4JYRc.nfRxnaogpSgQbDIQVNBLS_d state=0x555af423e6b8 s->prefetch_data=0 2024-03-04T17:45:31.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.167+0000 7fed69462980 10 manifest: total_size = 12 2024-03-04T17:45:31.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.167+0000 7fed69462980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4416903756600085270 2024-03-04T17:45:31.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.167+0000 7fed69462980 20 get_obj_state: octx=0x555af3f97310 obj=check-bucket:_:72vk4JYRc.nfRxnaogpSgQbDIQVNBLS_d state=0x555af423e6b8 s->prefetch_data=0 2024-03-04T17:45:31.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.167+0000 7fed69462980 20 prepare_atomic_modification: state is not atomic. state=0x555af423e6b8 2024-03-04T17:45:31.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.167+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.175+0000 7fec49ffb640 20 handle_completion(): completion ok for obj=d[72vk4JYRc.nfRxnaogpSgQbDIQVNBLS] 2024-03-04T17:45:31.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.179+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.179+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.182 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.179+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.183 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.183+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.184 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.183+0000 7fed69462980 20 get_obj_state: octx=0x555af3f97310 obj=check-bucket:e state=0x555af4276738 s->prefetch_data=0 2024-03-04T17:45:31.185 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.183+0000 7fed69462980 20 get_obj_state: setting s->obj_tag to 2jesmizr3pc6dnhevqdbxryk5lebh1kf 2024-03-04T17:45:31.185 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.183+0000 7fed69462980 20 get_obj_state_impl: setting s->olh_tag to 64mjcpruurup8rbavk5qpez34soyxgg1 2024-03-04T17:45:31.185 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.183+0000 7fed69462980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=64mjcpruurup8rbavk5qpez34soyxgg1 2024-03-04T17:45:31.188 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.187+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.191 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.191+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.192 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.191+0000 7fed69462980 20 olh_log_entry: epoch=2 op=2 key=e[] 2024-03-04T17:45:31.192 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.191+0000 7fed69462980 20 olh_log_entry: epoch=2 op=3 key=e[ypkt3cphIVtASMr3AQ.ujUgjjuJKjpb] 2024-03-04T17:45:31.192 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.191+0000 7fed69462980 20 get_obj_state: octx=0x555af3f97310 obj=check-bucket:_:ypkt3cphIVtASMr3AQ.ujUgjjuJKjpb_e state=0x555af423e6b8 s->prefetch_data=0 2024-03-04T17:45:31.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.191+0000 7fed69462980 10 manifest: total_size = 9 2024-03-04T17:45:31.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.191+0000 7fed69462980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.6890209124955312252 2024-03-04T17:45:31.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.191+0000 7fed69462980 20 get_obj_state: octx=0x555af3f97310 obj=check-bucket:_:ypkt3cphIVtASMr3AQ.ujUgjjuJKjpb_e state=0x555af423e6b8 s->prefetch_data=0 2024-03-04T17:45:31.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.191+0000 7fed69462980 20 prepare_atomic_modification: state is not atomic. state=0x555af423e6b8 2024-03-04T17:45:31.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.191+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.203+0000 7fed69462980 20 get_obj_state: octx=0x555af3f97310 obj=check-bucket:e state=0x555af4276738 s->prefetch_data=0 2024-03-04T17:45:31.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.203+0000 7fed69462980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=64mjcpruurup8rbavk5qpez34soyxgg1 2024-03-04T17:45:31.203 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.203+0000 7fec49ffb640 20 handle_completion(): completion ok for obj=e[ypkt3cphIVtASMr3AQ.ujUgjjuJKjpb] 2024-03-04T17:45:31.207 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.207+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.211+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.211+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.211+0000 7fed69462980 20 get_obj_state: octx=0x555af3f97310 obj=check-bucket:f state=0x555af4243568 s->prefetch_data=0 2024-03-04T17:45:31.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.211+0000 7fed69462980 20 get_obj_state: setting s->obj_tag to peuqez4dsm04yi0daa8oomby6cypyhm1 2024-03-04T17:45:31.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.211+0000 7fed69462980 20 get_obj_state_impl: setting s->olh_tag to xkx24h70elg8psa098wmj14v4kdrwewq 2024-03-04T17:45:31.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.215+0000 7fed69462980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=xkx24h70elg8psa098wmj14v4kdrwewq 2024-03-04T17:45:31.217 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.215+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.221 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.219+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.222 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.219+0000 7fed69462980 20 olh_log_entry: epoch=2 op=2 key=f[] 2024-03-04T17:45:31.222 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.219+0000 7fed69462980 20 olh_log_entry: epoch=2 op=3 key=f[olhtBX1jfbZ1fU9.ROc-p0gyZx7KCIF] 2024-03-04T17:45:31.222 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.219+0000 7fed69462980 20 get_obj_state: octx=0x555af3f97310 obj=check-bucket:_:olhtBX1jfbZ1fU9.ROc-p0gyZx7KCIF_f state=0x555af423e6b8 s->prefetch_data=0 2024-03-04T17:45:31.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.223+0000 7fed69462980 10 manifest: total_size = 11 2024-03-04T17:45:31.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.223+0000 7fed69462980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7479444231358880983 2024-03-04T17:45:31.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.223+0000 7fed69462980 20 get_obj_state: octx=0x555af3f97310 obj=check-bucket:_:olhtBX1jfbZ1fU9.ROc-p0gyZx7KCIF_f state=0x555af423e6b8 s->prefetch_data=0 2024-03-04T17:45:31.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.223+0000 7fed69462980 20 prepare_atomic_modification: state is not atomic. state=0x555af423e6b8 2024-03-04T17:45:31.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.223+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.232 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.231+0000 7fec49ffb640 20 handle_completion(): completion ok for obj=f[olhtBX1jfbZ1fU9.ROc-p0gyZx7KCIF] 2024-03-04T17:45:31.232 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.231+0000 7fed69462980 20 get_obj_state: octx=0x555af3f97310 obj=check-bucket:f state=0x555af4243568 s->prefetch_data=0 2024-03-04T17:45:31.233 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.231+0000 7fed69462980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=xkx24h70elg8psa098wmj14v4kdrwewq 2024-03-04T17:45:31.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.239+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.239+0000 7fed69462980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2.0 2024-03-04T17:45:31.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.239+0000 7fed69462980 1 NOTICE: finished shard 0 (4 entries removed) 2024-03-04T17:45:31.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.239+0000 7fed69462980 1 NOTICE: finished all shards (4 entries removed) 2024-03-04T17:45:31.244 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.243+0000 7fec1d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:31.250 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.247+0000 7fed69462980 20 remove_watcher() i=0 2024-03-04T17:45:31.250 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.247+0000 7fed69462980 2 removed watcher, disabling cache 2024-03-04T17:45:31.252 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.251+0000 7fed69462980 20 remove_watcher() i=1 2024-03-04T17:45:31.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.255+0000 7fed69462980 20 remove_watcher() i=2 2024-03-04T17:45:31.259 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.255+0000 7fed69462980 20 remove_watcher() i=3 2024-03-04T17:45:31.260 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.259+0000 7fed69462980 20 remove_watcher() i=4 2024-03-04T17:45:31.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.263+0000 7fed69462980 20 remove_watcher() i=5 2024-03-04T17:45:31.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.263+0000 7fed69462980 20 remove_watcher() i=6 2024-03-04T17:45:31.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.267+0000 7fed69462980 20 remove_watcher() i=7 2024-03-04T17:45:31.295 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:31.295 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:31.364 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:31.364 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:31.810 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket check-bucket 2024-03-04T17:45:31.843 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:45:31.843 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:45:31.886 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.887+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.887+0000 7fb366d33980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.887+0000 7fb366d33980 20 realm 2024-03-04T17:45:31.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.887+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.887+0000 7fb366d33980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.887+0000 7fb366d33980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:45:31.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.887+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.887+0000 7fb366d33980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:31.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.887+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.891+0000 7fb366d33980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:31.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.891+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:31.891 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.891+0000 7fb366d33980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:45:31.891 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.891+0000 7fb366d33980 20 searching for the correct realm 2024-03-04T17:45:31.897 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:31.895+0000 7fb36504c640 0 -- 172.21.15.137:0/1846751207 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fb35c059020 msgr2=0x7fb35c057380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:45:32.112 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.111+0000 7fb36504c640 0 -- 172.21.15.137:0/1846751207 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x561ab5062600 msgr2=0x561ab5064aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:45:32.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.119+0000 7fb366d33980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:45:32.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.119+0000 7fb366d33980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:45:32.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.119+0000 7fb366d33980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:45:32.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.119+0000 7fb366d33980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:45:32.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.119+0000 7fb366d33980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:45:32.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.119+0000 7fb366d33980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:45:32.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.119+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:32.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.119+0000 7fb366d33980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:32.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.119+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:32.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:45:32.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:32.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:45:32.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 20 zone default found 2024-03-04T17:45:32.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 4 Realm: () 2024-03-04T17:45:32.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:45:32.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:45:32.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:45:32.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 20 zonegroup default 2024-03-04T17:45:32.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:32.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:32.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:32.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:32.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:32.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:32.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 20 started sync module instance, tier type = 2024-03-04T17:45:32.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.123+0000 7fb366d33980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:45:32.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.147+0000 7fb366d33980 20 add_watcher() i=0 2024-03-04T17:45:32.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.147+0000 7fb366d33980 20 add_watcher() i=1 2024-03-04T17:45:32.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.147+0000 7fb366d33980 20 add_watcher() i=2 2024-03-04T17:45:32.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.147+0000 7fb366d33980 20 add_watcher() i=3 2024-03-04T17:45:32.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.147+0000 7fb366d33980 20 add_watcher() i=4 2024-03-04T17:45:32.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.147+0000 7fb366d33980 20 add_watcher() i=5 2024-03-04T17:45:32.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.147+0000 7fb366d33980 20 add_watcher() i=6 2024-03-04T17:45:32.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.151+0000 7fb366d33980 20 add_watcher() i=7 2024-03-04T17:45:32.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.151+0000 7fb366d33980 2 all 8 watchers are set, enabling cache 2024-03-04T17:45:32.156 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.155+0000 7fb21afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:32.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.155+0000 7fb366d33980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:45:32.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.155+0000 7fb366d33980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:45:32.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.155+0000 7fb366d33980 5 note: GC not initialized 2024-03-04T17:45:32.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.155+0000 7fb219fe3640 20 reqs_thread_entry: start 2024-03-04T17:45:32.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.155+0000 7fb366d33980 20 init_complete bucket index max shards: 1 2024-03-04T17:45:32.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.159+0000 7fb366d33980 20 Filter name: none 2024-03-04T17:45:32.159 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.159+0000 7fb18b7fe640 20 reqs_thread_entry: start 2024-03-04T17:45:32.159 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.159+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:32.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.159+0000 7fb366d33980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:32.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.159+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:32.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.159+0000 7fb366d33980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:32.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.159+0000 7fb366d33980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:32.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.159+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:32.162 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.159+0000 7fb366d33980 20 rados_obj.operate() r=0 bl.length=190 2024-03-04T17:45:32.162 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.159+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:32.163 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.163+0000 7fb366d33980 20 rados_obj.operate() r=0 bl.length=368 2024-03-04T17:45:32.163 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.163+0000 7fb366d33980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T17:45:32.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.163+0000 7fb366d33980 20 rados->read ofs=0 len=0 2024-03-04T17:45:32.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.163+0000 7fb366d33980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:45:32.166 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.163+0000 7fb21afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:45:32.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.171+0000 7fb366d33980 20 remove_watcher() i=0 2024-03-04T17:45:32.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.171+0000 7fb366d33980 2 removed watcher, disabling cache 2024-03-04T17:45:32.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.171+0000 7fb366d33980 20 remove_watcher() i=1 2024-03-04T17:45:32.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.175+0000 7fb366d33980 20 remove_watcher() i=2 2024-03-04T17:45:32.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.175+0000 7fb366d33980 20 remove_watcher() i=3 2024-03-04T17:45:32.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.179+0000 7fb366d33980 20 remove_watcher() i=4 2024-03-04T17:45:32.181 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.179+0000 7fb366d33980 20 remove_watcher() i=5 2024-03-04T17:45:32.184 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.183+0000 7fb366d33980 20 remove_watcher() i=6 2024-03-04T17:45:32.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:45:32.187+0000 7fb366d33980 20 remove_watcher() i=7 2024-03-04T17:45:32.203 INFO:tasks.workunit.client.0.smithi137.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2024-03-04T17:45:32.203 INFO:tasks.workunit.client.0.smithi137.stderr:Deleting bucket check-bucket 2024-03-04T17:45:32.205 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:32.205 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:32.234 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:45:32.234 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:45:32.551 INFO:tasks.workunit.client.0.smithi137.stderr:Completed bucket check tests 2024-03-04T17:45:32.552 INFO:tasks.workunit.client.0.smithi137.stdout:Could not connect to the endpoint URL: "http://localhost:80/" 2024-03-04T17:45:32.552 INFO:tasks.workunit.client.0.smithi137.stdout:Could not connect to the endpoint URL: "http://localhost:8000/" 2024-03-04T17:45:32.552 INFO:tasks.workunit.client.0.smithi137.stdout:connected to https://localhost:443 2024-03-04T17:45:32.652 INFO:tasks.workunit.client.0.smithi137.stderr:+ deactivate 2024-03-04T17:45:32.652 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-03-04T17:45:32.652 INFO:tasks.workunit.client.0.smithi137.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T17:45:32.652 INFO:tasks.workunit.client.0.smithi137.stderr:+ export PATH 2024-03-04T17:45:32.653 INFO:tasks.workunit.client.0.smithi137.stderr:+ unset _OLD_VIRTUAL_PATH 2024-03-04T17:45:32.653 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' -n '' ']' 2024-03-04T17:45:32.653 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T17:45:32.653 INFO:tasks.workunit.client.0.smithi137.stderr:+ hash -r 2024-03-04T17:45:32.653 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' -n '' ']' 2024-03-04T17:45:32.653 INFO:tasks.workunit.client.0.smithi137.stderr:+ unset VIRTUAL_ENV 2024-03-04T17:45:32.653 INFO:tasks.workunit.client.0.smithi137.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-03-04T17:45:32.653 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' '!' '' = nondestructive ']' 2024-03-04T17:45:32.653 INFO:tasks.workunit.client.0.smithi137.stderr:+ unset -f deactivate 2024-03-04T17:45:32.653 INFO:tasks.workunit.client.0.smithi137.stderr:+ echo OK. 2024-03-04T17:45:32.654 INFO:tasks.workunit.client.0.smithi137.stdout:OK. 2024-03-04T17:45:32.654 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:45:32.655 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:45:32.710 INFO:tasks.workunit:Stopping ['rgw/run-bucket-check.sh'] on client.0... 2024-03-04T17:45:32.710 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-04T17:45:33.132 DEBUG:teuthology.parallel:result is None 2024-03-04T17:45:33.132 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:45:33.145 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:45:33.145 DEBUG:teuthology.orchestra.run.smithi137:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:45:33.190 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:45:33.190 INFO:teuthology.run_tasks:Running task workunit... 2024-03-04T17:45:33.200 INFO:tasks.workunit:Pulling workunits from ref 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:45:33.200 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-04T17:45:33.200 DEBUG:teuthology.orchestra.run.smithi137:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:45:33.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:45:33.239 INFO:teuthology.orchestra.run.smithi137.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-04T17:45:33.239 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:45:33.286 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-04T17:45:33.287 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-04T17:45:33.335 INFO:tasks.workunit:timeout=3h 2024-03-04T17:45:33.335 INFO:tasks.workunit:cleanup=True 2024-03-04T17:45:33.335 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/pritha-srivastava/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:45:33.389 INFO:tasks.workunit.client.0.smithi137.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-04T17:46:33.139 INFO:tasks.workunit.client.0.smithi137.stderr:Note: switching to '3aec390336fdfdc8f927818b718cfb0361bd47c8'. 2024-03-04T17:46:33.139 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:46:33.139 INFO:tasks.workunit.client.0.smithi137.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T17:46:33.139 INFO:tasks.workunit.client.0.smithi137.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T17:46:33.139 INFO:tasks.workunit.client.0.smithi137.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T17:46:33.139 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:46:33.139 INFO:tasks.workunit.client.0.smithi137.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T17:46:33.139 INFO:tasks.workunit.client.0.smithi137.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T17:46:33.139 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:46:33.140 INFO:tasks.workunit.client.0.smithi137.stderr: git switch -c 2024-03-04T17:46:33.140 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:46:33.140 INFO:tasks.workunit.client.0.smithi137.stderr:Or undo this operation with: 2024-03-04T17:46:33.140 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:46:33.140 INFO:tasks.workunit.client.0.smithi137.stderr: git switch - 2024-03-04T17:46:33.140 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:46:33.140 INFO:tasks.workunit.client.0.smithi137.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T17:46:33.140 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:46:33.140 INFO:tasks.workunit.client.0.smithi137.stderr:HEAD is now at 3aec390336f qa/d4n: Start redis up before rgw 2024-03-04T17:46:33.145 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-04T17:46:33.194 INFO:tasks.workunit.client.0.smithi137.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T17:46:33.196 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T17:46:33.196 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T17:46:33.248 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T17:46:33.300 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T17:46:33.339 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T17:46:33.342 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T17:46:33.342 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T17:46:33.384 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T17:46:33.387 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:46:33.387 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-04T17:46:33.437 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2024-03-04T17:46:33.438 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2024-03-04T17:46:33.438 DEBUG:teuthology.orchestra.run.smithi137:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2024-03-04T17:46:33.498 INFO:tasks.workunit.client.0.smithi137.stdout:Running main() from gmock_main.cc 2024-03-04T17:46:33.499 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 11 tests from 1 test suite. 2024-03-04T17:46:33.499 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:46:33.499 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 11 tests from ClsLock 2024-03-04T17:46:33.499 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] ClsLock.TestMultiLocking 2024-03-04T17:46:35.302 INFO:tasks.workunit.client.0.smithi137.stdout:lock: mylock 2024-03-04T17:46:35.303 INFO:tasks.workunit.client.0.smithi137.stdout: lock_type: exclusive 2024-03-04T17:46:35.303 INFO:tasks.workunit.client.0.smithi137.stdout: tag: 2024-03-04T17:46:35.303 INFO:tasks.workunit.client.0.smithi137.stdout: lockers: 2024-03-04T17:46:35.303 INFO:tasks.workunit.client.0.smithi137.stdout: client.4988 expiration=2024-03-04T17:48:35.299412+0000 addr=v1:172.21.15.137:0/74292979 cookie= 2024-03-04T17:46:35.309 INFO:tasks.workunit.client.0.smithi137.stdout:lock: mylock 2024-03-04T17:46:35.309 INFO:tasks.workunit.client.0.smithi137.stdout: lock_type: shared 2024-03-04T17:46:35.309 INFO:tasks.workunit.client.0.smithi137.stdout: tag: 2024-03-04T17:46:35.309 INFO:tasks.workunit.client.0.smithi137.stdout: lockers: 2024-03-04T17:46:35.309 INFO:tasks.workunit.client.0.smithi137.stdout: client.4988 expiration=2024-03-04T17:48:35.309354+0000 addr=v1:172.21.15.137:0/74292979 cookie= 2024-03-04T17:46:35.309 INFO:tasks.workunit.client.0.smithi137.stdout: client.5133 expiration=0.000000 addr=v1:172.21.15.137:0/2504612964 cookie= 2024-03-04T17:46:35.311 INFO:tasks.workunit.client.0.smithi137.stdout:lock: mylock 2024-03-04T17:46:35.311 INFO:tasks.workunit.client.0.smithi137.stdout: lock_type: shared 2024-03-04T17:46:35.311 INFO:tasks.workunit.client.0.smithi137.stdout: tag: 2024-03-04T17:46:35.311 INFO:tasks.workunit.client.0.smithi137.stdout: lockers: 2024-03-04T17:46:35.311 INFO:tasks.workunit.client.0.smithi137.stdout: client.5133 expiration=0.000000 addr=v1:172.21.15.137:0/2504612964 cookie= 2024-03-04T17:46:36.252 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] ClsLock.TestMultiLocking (2754 ms) 2024-03-04T17:46:36.253 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] ClsLock.TestMeta 2024-03-04T17:46:38.308 INFO:tasks.workunit.client.0.smithi137.stdout:lock: mylock 2024-03-04T17:46:38.308 INFO:tasks.workunit.client.0.smithi137.stdout: lock_type: shared 2024-03-04T17:46:38.308 INFO:tasks.workunit.client.0.smithi137.stdout: tag: 2024-03-04T17:46:38.308 INFO:tasks.workunit.client.0.smithi137.stdout: lockers: 2024-03-04T17:46:38.308 INFO:tasks.workunit.client.0.smithi137.stdout: client.5006 expiration=0.000000 addr=v1:172.21.15.137:0/26818118 cookie= 2024-03-04T17:46:38.310 INFO:tasks.workunit.client.0.smithi137.stdout:lock: mylock 2024-03-04T17:46:38.310 INFO:tasks.workunit.client.0.smithi137.stdout: lock_type: exclusive 2024-03-04T17:46:38.310 INFO:tasks.workunit.client.0.smithi137.stdout: tag: new_tag 2024-03-04T17:46:38.310 INFO:tasks.workunit.client.0.smithi137.stdout: lockers: 2024-03-04T17:46:38.310 INFO:tasks.workunit.client.0.smithi137.stdout: client.5000 expiration=0.000000 addr=v1:172.21.15.137:0/3430374975 cookie= 2024-03-04T17:46:39.319 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] ClsLock.TestMeta (3066 ms) 2024-03-04T17:46:39.319 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] ClsLock.TestCookie 2024-03-04T17:46:41.350 INFO:tasks.workunit.client.0.smithi137.stdout:lock: mylock 2024-03-04T17:46:41.351 INFO:tasks.workunit.client.0.smithi137.stdout: lock_type: exclusive 2024-03-04T17:46:41.351 INFO:tasks.workunit.client.0.smithi137.stdout: tag: 2024-03-04T17:46:41.351 INFO:tasks.workunit.client.0.smithi137.stdout: lockers: 2024-03-04T17:46:41.357 INFO:tasks.workunit.client.0.smithi137.stdout:lock: mylock 2024-03-04T17:46:41.357 INFO:tasks.workunit.client.0.smithi137.stdout: lock_type: shared 2024-03-04T17:46:41.357 INFO:tasks.workunit.client.0.smithi137.stdout: tag: 2024-03-04T17:46:41.357 INFO:tasks.workunit.client.0.smithi137.stdout: lockers: 2024-03-04T17:46:41.357 INFO:tasks.workunit.client.0.smithi137.stdout: client.5148 expiration=0.000000 addr=v1:172.21.15.137:0/2642912652 cookie= 2024-03-04T17:46:41.357 INFO:tasks.workunit.client.0.smithi137.stdout: client.5148 expiration=0.000000 addr=v1:172.21.15.137:0/2642912652 cookie=new cookie 2024-03-04T17:46:42.336 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] ClsLock.TestCookie (3017 ms) 2024-03-04T17:46:42.336 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] ClsLock.TestMultipleLocks 2024-03-04T17:46:45.350 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] ClsLock.TestMultipleLocks (3009 ms) 2024-03-04T17:46:45.351 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] ClsLock.TestLockDuration 2024-03-04T17:46:52.494 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] ClsLock.TestLockDuration (7149 ms) 2024-03-04T17:46:52.495 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] ClsLock.TestAssertLocked 2024-03-04T17:46:55.544 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] ClsLock.TestAssertLocked (3049 ms) 2024-03-04T17:46:55.544 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] ClsLock.TestSetCookie 2024-03-04T17:46:58.549 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] ClsLock.TestSetCookie (3005 ms) 2024-03-04T17:46:58.549 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] ClsLock.TestRenew 2024-03-04T17:47:19.149 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] ClsLock.TestRenew (20600 ms) 2024-03-04T17:47:19.149 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2024-03-04T17:47:26.219 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (7068 ms) 2024-03-04T17:47:26.219 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2024-03-04T17:47:34.130 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (7912 ms) 2024-03-04T17:47:34.130 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2024-03-04T17:47:42.200 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (8063 ms) 2024-03-04T17:47:42.200 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 11 tests from ClsLock (68693 ms total) 2024-03-04T17:47:42.200 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:47:42.200 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:47:42.201 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 11 tests from 1 test suite ran. (68693 ms total) 2024-03-04T17:47:42.201 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 11 tests. 2024-03-04T17:47:42.201 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:47:42.201 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:47:42.253 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2024-03-04T17:47:42.254 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2024-03-04T17:47:42.254 DEBUG:teuthology.orchestra.run.smithi137:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2024-03-04T17:47:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:Running main() from gmock_main.cc 2024-03-04T17:47:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 4 tests from 1 test suite. 2024-03-04T17:47:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:47:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 4 tests from cls_log 2024-03-04T17:47:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_log.test_log_add_same_time 2024-03-04T17:47:45.217 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_log.test_log_add_same_time (2896 ms) 2024-03-04T17:47:45.217 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_log.test_log_add_different_time 2024-03-04T17:47:48.280 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_log.test_log_add_different_time (3063 ms) 2024-03-04T17:47:48.280 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_log.trim_by_time 2024-03-04T17:47:51.293 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_log.trim_by_time (3008 ms) 2024-03-04T17:47:51.293 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_log.trim_by_marker 2024-03-04T17:47:54.357 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_log.trim_by_marker (3069 ms) 2024-03-04T17:47:54.357 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 4 tests from cls_log (12036 ms total) 2024-03-04T17:47:54.357 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:47:54.357 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:47:54.357 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 4 tests from 1 test suite ran. (12036 ms total) 2024-03-04T17:47:54.357 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 4 tests. 2024-03-04T17:47:54.362 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:47:54.362 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:47:54.421 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2024-03-04T17:47:54.422 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2024-03-04T17:47:54.423 DEBUG:teuthology.orchestra.run.smithi137:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2024-03-04T17:47:54.492 INFO:tasks.workunit.client.0.smithi137.stdout:Running main() from gmock_main.cc 2024-03-04T17:47:54.492 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 10 tests from 1 test suite. 2024-03-04T17:47:54.492 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:47:54.492 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 10 tests from cls_refcount 2024-03-04T17:47:54.492 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_refcount.test_implicit 2024-03-04T17:47:57.369 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_refcount.test_implicit (2877 ms) 2024-03-04T17:47:57.369 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_refcount.test_implicit_idempotent 2024-03-04T17:48:00.435 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_refcount.test_implicit_idempotent (3065 ms) 2024-03-04T17:48:00.435 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_refcount.test_put_snap 2024-03-04T17:48:09.189 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_refcount.test_put_snap (8751 ms) 2024-03-04T17:48:09.189 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_refcount.test_explicit 2024-03-04T17:48:12.195 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_refcount.test_explicit (3010 ms) 2024-03-04T17:48:12.195 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_refcount.set 2024-03-04T17:48:15.252 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_refcount.set (3057 ms) 2024-03-04T17:48:15.252 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_refcount.test_implicit_ec 2024-03-04T17:48:22.316 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_refcount.test_implicit_ec (7065 ms) 2024-03-04T17:48:22.317 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_refcount.test_implicit_idempotent_ec 2024-03-04T17:48:29.380 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_refcount.test_implicit_idempotent_ec (7061 ms) 2024-03-04T17:48:29.381 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_refcount.test_put_snap_ec 2024-03-04T17:48:41.501 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_refcount.test_put_snap_ec (12124 ms) 2024-03-04T17:48:41.502 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_refcount.test_explicit_ec 2024-03-04T17:48:48.600 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_refcount.test_explicit_ec (7098 ms) 2024-03-04T17:48:48.601 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_refcount.set_ec 2024-03-04T17:48:55.682 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_refcount.set_ec (7082 ms) 2024-03-04T17:48:55.682 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 10 tests from cls_refcount (61190 ms total) 2024-03-04T17:48:55.682 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:48:55.682 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:48:55.683 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 10 tests from 1 test suite ran. (61190 ms total) 2024-03-04T17:48:55.683 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 10 tests. 2024-03-04T17:48:55.687 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:48:55.688 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:48:55.742 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2024-03-04T17:48:55.743 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2024-03-04T17:48:55.743 DEBUG:teuthology.orchestra.run.smithi137:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2024-03-04T17:48:55.806 INFO:tasks.workunit.client.0.smithi137.stdout:Running main() from gmock_main.cc 2024-03-04T17:48:55.807 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 16 tests from 1 test suite. 2024-03-04T17:48:55.807 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:48:55.807 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 16 tests from cls_rgw 2024-03-04T17:48:57.691 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.index_basic 2024-03-04T17:48:57.803 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.index_basic (113 ms) 2024-03-04T17:48:57.804 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2024-03-04T17:48:57.880 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (77 ms) 2024-03-04T17:48:57.880 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.index_remove_object 2024-03-04T17:48:57.979 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.index_remove_object (99 ms) 2024-03-04T17:48:57.979 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.index_suggest 2024-03-04T17:49:00.371 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.index_suggest (2391 ms) 2024-03-04T17:49:00.371 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.index_suggest_complete 2024-03-04T17:49:00.386 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.index_suggest_complete (15 ms) 2024-03-04T17:49:00.386 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.index_list 2024-03-04T17:49:00.439 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.index_list (54 ms) 2024-03-04T17:49:00.440 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.index_list_delimited 2024-03-04T17:49:59.544 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.index_list_delimited (59104 ms) 2024-03-04T17:49:59.544 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.bi_list 2024-03-04T17:49:59.771 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.bi_list (228 ms) 2024-03-04T17:49:59.771 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.gc_set 2024-03-04T17:49:59.800 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.gc_set (29 ms) 2024-03-04T17:49:59.800 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.gc_list 2024-03-04T17:49:59.831 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.gc_list (31 ms) 2024-03-04T17:49:59.831 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.gc_defer 2024-03-04T17:50:06.306 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.gc_defer (6475 ms) 2024-03-04T17:50:06.306 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.usage_basic 2024-03-04T17:50:06.521 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.usage_basic (215 ms) 2024-03-04T17:50:06.521 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2024-03-04T17:50:06.524 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.usage_clear_no_obj (3 ms) 2024-03-04T17:50:06.524 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.usage_clear 2024-03-04T17:50:06.744 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.usage_clear (220 ms) 2024-03-04T17:50:06.744 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.bi_log_trim 2024-03-04T17:50:06.880 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.bi_log_trim (136 ms) 2024-03-04T17:50:06.880 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw.index_racing_removes 2024-03-04T17:50:06.933 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw.index_racing_removes (53 ms) 2024-03-04T17:50:07.317 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 16 tests from cls_rgw (69243 ms total) 2024-03-04T17:50:07.317 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:50:07.317 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:50:07.318 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 16 tests from 1 test suite ran. (71511 ms total) 2024-03-04T17:50:07.318 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 16 tests. 2024-03-04T17:50:07.325 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:50:07.326 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:50:07.380 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_gc.sh on client.0... 2024-03-04T17:50:07.381 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_gc.sh... 2024-03-04T17:50:07.382 DEBUG:teuthology.orchestra.run.smithi137:workunit test cls/test_cls_rgw_gc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_gc.sh 2024-03-04T17:50:07.447 INFO:tasks.workunit.client.0.smithi137.stdout:Running main() from gmock_main.cc 2024-03-04T17:50:07.447 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 9 tests from 1 test suite. 2024-03-04T17:50:07.447 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:50:07.447 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 9 tests from cls_rgw_gc 2024-03-04T17:50:07.447 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw_gc.init 2024-03-04T17:50:09.316 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw_gc.init (1869 ms) 2024-03-04T17:50:09.316 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops1 2024-03-04T17:50:09.368 INFO:tasks.workunit.client.0.smithi137.stderr:[ ] list info tag = chain-0 2024-03-04T17:50:09.369 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw_gc.gc_queue_ops1 (52 ms) 2024-03-04T17:50:09.369 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops2 2024-03-04T17:50:09.383 INFO:tasks.workunit.client.0.smithi137.stderr:[ ] next_marker is: = 0/4263 2024-03-04T17:50:09.384 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw_gc.gc_queue_ops2 (16 ms) 2024-03-04T17:50:09.384 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops5 2024-03-04T17:50:09.405 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw_gc.gc_queue_ops5 (21 ms) 2024-03-04T17:50:09.405 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops6 2024-03-04T17:50:09.421 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw_gc.gc_queue_ops6 (17 ms) 2024-03-04T17:50:09.421 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops7 2024-03-04T17:50:09.439 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw_gc.gc_queue_ops7 (17 ms) 2024-03-04T17:50:09.439 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops8 2024-03-04T17:50:09.460 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw_gc.gc_queue_ops8 (21 ms) 2024-03-04T17:50:09.460 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops9 2024-03-04T17:50:09.477 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw_gc.gc_queue_ops9 (17 ms) 2024-03-04T17:50:09.477 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw_gc.finalize 2024-03-04T17:50:10.328 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw_gc.finalize (852 ms) 2024-03-04T17:50:10.329 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 9 tests from cls_rgw_gc (2882 ms total) 2024-03-04T17:50:10.329 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:50:10.329 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:50:10.329 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 9 tests from 1 test suite ran. (2882 ms total) 2024-03-04T17:50:10.329 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 9 tests. 2024-03-04T17:50:10.334 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:50:10.334 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:50:10.389 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_stats.sh on client.0... 2024-03-04T17:50:10.390 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_stats.sh... 2024-03-04T17:50:10.390 DEBUG:teuthology.orchestra.run.smithi137:workunit test cls/test_cls_rgw_stats.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_stats.sh 2024-03-04T17:50:10.465 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:50:10.465 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:50:10.465 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 1 test from 1 test suite. 2024-03-04T17:50:10.465 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:50:12.357 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 1 test from cls_rgw_stats 2024-03-04T17:50:12.358 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] cls_rgw_stats.simulate 2024-03-04T17:50:12.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.384+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=3JGXO113LPO7 type=write size=304930 2024-03-04T17:50:12.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.388+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=PJG9GHZM2RMR type=write size=1044084 2024-03-04T17:50:12.393 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.392+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=LL4CCW9PKGTV type=del size=355358 2024-03-04T17:50:12.399 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.400+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~FH3FDM1AZI4B.0 size=1048576 2024-03-04T17:50:12.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.400+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~FH3FDM1AZI4B.1 size=1048576 2024-03-04T17:50:12.404 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.404+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~FH3FDM1AZI4B.2 size=1972 2024-03-04T17:50:12.405 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.404+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=FH3FDM1AZI4B tag=D76ZLMJ3Q7RB type=write size=3147700 2024-03-04T17:50:12.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.412+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~R8C9866T81LA.0 size=1048576 2024-03-04T17:50:12.414 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.412+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~R8C9866T81LA.1 size=1048576 2024-03-04T17:50:12.416 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.416+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~R8C9866T81LA.2 size=15445 2024-03-04T17:50:12.416 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.416+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=R8C9866T81LA tag=8C9HF4ED2GNR type=write size=3161173 2024-03-04T17:50:12.423 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.420+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~UA7EI3HOSVA9.0 size=1048576 2024-03-04T17:50:12.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.424+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~UA7EI3HOSVA9.1 size=1048576 2024-03-04T17:50:12.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.440+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~UA7EI3HOSVA9.2 size=335207 2024-03-04T17:50:12.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.440+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=UA7EI3HOSVA9 tag=N25DBM0II32Y type=write size=3480935 2024-03-04T17:50:12.444 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.444+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=997YI1QBPHNM type=del size=3004635 2024-03-04T17:50:12.451 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.448+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~GWDG34DV6XMN.0 size=1048576 2024-03-04T17:50:12.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.452+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~GWDG34DV6XMN.1 size=1048576 2024-03-04T17:50:12.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.456+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~GWDG34DV6XMN.2 size=503289 2024-03-04T17:50:12.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.456+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=GWDG34DV6XMN tag=41QP4K25VSZZ type=write size=3649017 2024-03-04T17:50:12.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.460+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=29HGQUKLTKGT type=del size=3645397 2024-03-04T17:50:12.468 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.468+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~J8XLFSLYTZHR.0 size=1048576 2024-03-04T17:50:12.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.468+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~J8XLFSLYTZHR.1 size=1048576 2024-03-04T17:50:12.474 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.472+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~J8XLFSLYTZHR.2 size=289157 2024-03-04T17:50:12.474 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.472+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=J8XLFSLYTZHR tag=9FBV85KNY2S2 type=write size=3434885 2024-03-04T17:50:12.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.476+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=7KK2I3MVL2DE type=del size=2466622 2024-03-04T17:50:12.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.480+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=0E8IEUS6XDFW type=write size=895695 2024-03-04T17:50:12.488 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.488+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~183WL2J73L8K.0 size=1048576 2024-03-04T17:50:12.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.488+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~183WL2J73L8K.1 size=1048576 2024-03-04T17:50:12.494 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.492+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~183WL2J73L8K.2 size=435154 2024-03-04T17:50:12.494 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.492+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=183WL2J73L8K tag=1BCKQUZAMDX1 type=write size=3580882 2024-03-04T17:50:12.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.496+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=8QV2F30SZ0U7 type=del size=2088046 2024-03-04T17:50:12.502 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.500+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=0C37VL7UDFP7 type=write size=961589 2024-03-04T17:50:12.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.504+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=70MUYIFE1G8M type=del size=1511652 2024-03-04T17:50:12.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.508+0000 7fe960f568c0 1 < completed write operation key=2MKQA5JYWU8L tag=3JGXO113LPO7 type=write size=304930 2024-03-04T17:50:12.514 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.512+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=WAIA33FDF238 type=write size=934986 2024-03-04T17:50:12.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.524+0000 7fe960f568c0 1 < suggested remove operation key=3YTAYUQLLUKY tag=PJG9GHZM2RMR type=write 2024-03-04T17:50:12.530 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.528+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~SMCBL8BIP5NZ.0 size=1048576 2024-03-04T17:50:12.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.532+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~SMCBL8BIP5NZ.1 size=1048576 2024-03-04T17:50:12.535 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.536+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~SMCBL8BIP5NZ.2 size=46498 2024-03-04T17:50:12.536 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.536+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=SMCBL8BIP5NZ tag=PFO3C5IEIQLJ type=write size=3192226 2024-03-04T17:50:12.539 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.540+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=LL4CCW9PKGTV type=del 2024-03-04T17:50:12.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.540+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=QD73VCW17GKA type=del size=1214148 2024-03-04T17:50:12.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.544+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~FH3FDM1AZI4B.0 size=1048576 2024-03-04T17:50:12.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.548+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~FH3FDM1AZI4B.1 size=1048576 2024-03-04T17:50:12.553 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.552+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~FH3FDM1AZI4B.2 size=1972 2024-03-04T17:50:12.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.560+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=FH3FDM1AZI4B tag=D76ZLMJ3Q7RB type=write size=3147700 2024-03-04T17:50:12.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.564+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=2W5APW2RXECK type=del size=3976967 2024-03-04T17:50:12.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.568+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~R8C9866T81LA.0 size=1048576 2024-03-04T17:50:12.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.572+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~R8C9866T81LA.1 size=1048576 2024-03-04T17:50:12.574 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.572+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~R8C9866T81LA.2 size=15445 2024-03-04T17:50:12.581 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.580+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=R8C9866T81LA tag=8C9HF4ED2GNR type=write size=3161173 2024-03-04T17:50:12.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.584+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=BYB4J9EQVBOM type=del size=815131 2024-03-04T17:50:12.588 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.588+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~UA7EI3HOSVA9.0 size=1048576 2024-03-04T17:50:12.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.588+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~UA7EI3HOSVA9.1 size=1048576 2024-03-04T17:50:12.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.592+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~UA7EI3HOSVA9.2 size=335207 2024-03-04T17:50:12.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.600+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=UA7EI3HOSVA9 tag=N25DBM0II32Y type=write size=3480935 2024-03-04T17:50:12.604 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.604+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=RI7F7C1H8ZCR type=del size=3864937 2024-03-04T17:50:12.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.608+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=997YI1QBPHNM type=del 2024-03-04T17:50:12.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.612+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=VSTW2F6J2VNG type=del size=2648229 2024-03-04T17:50:12.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.616+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~GWDG34DV6XMN.0 size=1048576 2024-03-04T17:50:12.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.616+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~GWDG34DV6XMN.1 size=1048576 2024-03-04T17:50:12.621 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.620+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~GWDG34DV6XMN.2 size=503289 2024-03-04T17:50:12.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.628+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=GWDG34DV6XMN tag=41QP4K25VSZZ type=write size=3649017 2024-03-04T17:50:12.632 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.632+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=45EVWNMB3LHM type=del size=1227758 2024-03-04T17:50:12.639 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.636+0000 7fe960f568c0 1 < suggested update operation key=JSO6WIYMPCF6 tag=29HGQUKLTKGT type=del size=3645397 2024-03-04T17:50:12.643 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.640+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=XMJWCOQ449I4 type=del size=1321162 2024-03-04T17:50:12.646 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.644+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~J8XLFSLYTZHR.0 size=1048576 2024-03-04T17:50:12.649 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.648+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~J8XLFSLYTZHR.1 size=1048576 2024-03-04T17:50:12.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.652+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~J8XLFSLYTZHR.2 size=289157 2024-03-04T17:50:12.658 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.656+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=J8XLFSLYTZHR tag=9FBV85KNY2S2 type=write size=3434885 2024-03-04T17:50:12.661 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.660+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=U1RENAT3SEUY type=del size=1793818 2024-03-04T17:50:12.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.664+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=7KK2I3MVL2DE type=del 2024-03-04T17:50:12.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.672+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~489Z65IM0NAW.0 size=545133 2024-03-04T17:50:12.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.672+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=489Z65IM0NAW tag=X1NXHXM3DIQY type=write size=1593709 2024-03-04T17:50:12.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.676+0000 7fe960f568c0 1 < completed write operation key=QGLN27WH1N2O tag=0E8IEUS6XDFW type=write size=895695 2024-03-04T17:50:12.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.680+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=TGDWEC6R2O0V type=del size=2573269 2024-03-04T17:50:12.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.684+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~183WL2J73L8K.0 size=1048576 2024-03-04T17:50:12.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.688+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~183WL2J73L8K.1 size=1048576 2024-03-04T17:50:12.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.688+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~183WL2J73L8K.2 size=435154 2024-03-04T17:50:12.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.696+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=183WL2J73L8K tag=1BCKQUZAMDX1 type=write size=3580882 2024-03-04T17:50:12.702 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.700+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=AKNZIKK4Y5S4 type=del size=4165096 2024-03-04T17:50:12.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.704+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=8QV2F30SZ0U7 type=del 2024-03-04T17:50:12.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.712+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~PF9MIOAVCVM0.0 size=325490 2024-03-04T17:50:12.714 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.712+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=PF9MIOAVCVM0 tag=QQU9Q8QA5DBD type=write size=1374066 2024-03-04T17:50:12.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.720+0000 7fe960f568c0 1 < suggested remove operation key=R6ZDVOUE9NXJ tag=0C37VL7UDFP7 type=write 2024-03-04T17:50:12.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.728+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~T27QRZBI0AN3.0 size=1048576 2024-03-04T17:50:12.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.732+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~T27QRZBI0AN3.1 size=531249 2024-03-04T17:50:12.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.732+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=T27QRZBI0AN3 tag=IY583FKK4ALN type=write size=2628401 2024-03-04T17:50:12.739 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.736+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=70MUYIFE1G8M type=del 2024-03-04T17:50:12.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.744+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~I3J22J5SU6XD.0 size=1048576 2024-03-04T17:50:12.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.748+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~I3J22J5SU6XD.1 size=1048576 2024-03-04T17:50:12.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.752+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~I3J22J5SU6XD.2 size=608522 2024-03-04T17:50:12.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.752+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=I3J22J5SU6XD tag=RI40A6HA9VN1 type=write size=3754250 2024-03-04T17:50:12.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.760+0000 7fe960f568c0 1 < suggested remove operation key=7Z7BPZ8XGBHZ tag=WAIA33FDF238 type=write 2024-03-04T17:50:12.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.772+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~623DF0SHYRWS.0 size=622831 2024-03-04T17:50:12.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.772+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=623DF0SHYRWS tag=X9Z9QKZPO7PB type=write size=1671407 2024-03-04T17:50:12.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.772+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~SMCBL8BIP5NZ.0 size=1048576 2024-03-04T17:50:12.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.772+0000 7fe960f568c0 1 < canceled part key=_multipart_78N8HHC4O4CG.2~SMCBL8BIP5NZ.1 size=1048576 2024-03-04T17:50:12.776 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.776+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~SMCBL8BIP5NZ.2 size=46498 2024-03-04T17:50:12.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.784+0000 7fe960f568c0 1 < canceled multipart upload key=78N8HHC4O4CG upload=SMCBL8BIP5NZ tag=PFO3C5IEIQLJ type=write size=3192226 2024-03-04T17:50:12.788 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.788+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=WM5C8J3ZPQY5 type=write size=466806 2024-03-04T17:50:12.793 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.792+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=QD73VCW17GKA type=del 2024-03-04T17:50:12.797 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.796+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=6XG8G81CAFI9 type=del size=383780 2024-03-04T17:50:12.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.804+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=2W5APW2RXECK type=del 2024-03-04T17:50:12.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.804+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=BDQOOQN2LKU2 type=del size=158776 2024-03-04T17:50:12.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.808+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=BYB4J9EQVBOM type=del 2024-03-04T17:50:12.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.812+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=NTMQBEU0TZV4 type=del size=2745434 2024-03-04T17:50:12.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.816+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=RI7F7C1H8ZCR type=del 2024-03-04T17:50:12.823 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.820+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=OF3KW3IZCG26 type=write size=1005912 2024-03-04T17:50:12.827 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.824+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=VSTW2F6J2VNG type=del 2024-03-04T17:50:12.835 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.832+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~YFQMSS6G9Q55.0 size=1048576 2024-03-04T17:50:12.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.836+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~YFQMSS6G9Q55.1 size=1048576 2024-03-04T17:50:12.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.840+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~YFQMSS6G9Q55.2 size=808890 2024-03-04T17:50:12.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.840+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=YFQMSS6G9Q55 tag=U0UWONMV3FJ7 type=write size=3954618 2024-03-04T17:50:12.844 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.844+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=45EVWNMB3LHM type=del 2024-03-04T17:50:12.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.848+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=X3OGV6QZOPWY type=write size=128031 2024-03-04T17:50:12.852 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.852+0000 7fe960f568c0 1 < canceled operation key=2MKQA5JYWU8L tag=XMJWCOQ449I4 type=del size=1321162 2024-03-04T17:50:12.859 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.856+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~4XD9YX59F1D6.0 size=1048576 2024-03-04T17:50:12.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.860+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~4XD9YX59F1D6.1 size=317727 2024-03-04T17:50:12.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.860+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=4XD9YX59F1D6 tag=S6F0BQFBHE6F type=write size=2414879 2024-03-04T17:50:12.866 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.864+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=U1RENAT3SEUY type=del 2024-03-04T17:50:12.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.868+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=MTAOXH8QQDEJ type=del size=3975345 2024-03-04T17:50:12.874 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.872+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~489Z65IM0NAW.0 size=545133 2024-03-04T17:50:12.880 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.880+0000 7fe960f568c0 1 < completed multipart upload key=ASC735ZCBLCG upload=489Z65IM0NAW tag=X1NXHXM3DIQY type=write size=1593709 2024-03-04T17:50:12.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.884+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=ZETGE96F6GQF type=write size=382182 2024-03-04T17:50:12.887 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.888+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=TGDWEC6R2O0V type=del 2024-03-04T17:50:12.891 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.888+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=3FPRBY8HGR0T type=del size=1622754 2024-03-04T17:50:12.895 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.892+0000 7fe960f568c0 -1 < failed to complete operation key=R6ZDVOUE9NXJ tag=AKNZIKK4Y5S4 type=del size=4165096: (22) Invalid argument 2024-03-04T17:50:12.899 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.900+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=IQAOCOVWVZVC type=del size=3240204 2024-03-04T17:50:12.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.904+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~PF9MIOAVCVM0.0 size=325490 2024-03-04T17:50:12.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.908+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=PF9MIOAVCVM0 tag=QQU9Q8QA5DBD type=write size=1374066 2024-03-04T17:50:12.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.912+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=NFYMUTGPE48J type=del size=2976488 2024-03-04T17:50:12.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.916+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~T27QRZBI0AN3.0 size=1048576 2024-03-04T17:50:12.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.916+0000 7fe960f568c0 1 < canceled part key=_multipart_2KZ4IFW54C5B.2~T27QRZBI0AN3.1 size=531249 2024-03-04T17:50:12.925 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.924+0000 7fe960f568c0 1 < canceled multipart upload key=2KZ4IFW54C5B upload=T27QRZBI0AN3 tag=IY583FKK4ALN type=write size=2628401 2024-03-04T17:50:12.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.932+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~25LEIKP4TUHT.0 size=1048576 2024-03-04T17:50:12.935 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.932+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~25LEIKP4TUHT.1 size=1048576 2024-03-04T17:50:12.938 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.936+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~25LEIKP4TUHT.2 size=733278 2024-03-04T17:50:12.938 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.936+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=25LEIKP4TUHT tag=GCX8F83G3I3T type=write size=3879006 2024-03-04T17:50:12.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.940+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~I3J22J5SU6XD.0 size=1048576 2024-03-04T17:50:12.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.940+0000 7fe960f568c0 1 < canceled part key=_multipart_BFF9OD00TU54.2~I3J22J5SU6XD.1 size=1048576 2024-03-04T17:50:12.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.944+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~I3J22J5SU6XD.2 size=608522 2024-03-04T17:50:12.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.952+0000 7fe960f568c0 1 < canceled multipart upload key=BFF9OD00TU54 upload=I3J22J5SU6XD tag=RI40A6HA9VN1 type=write size=3754250 2024-03-04T17:50:12.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.956+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=6AJLXC6X6G4V type=del size=697343 2024-03-04T17:50:12.960 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.960+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~623DF0SHYRWS.0 size=622831 2024-03-04T17:50:12.966 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.964+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=623DF0SHYRWS tag=X9Z9QKZPO7PB type=write size=1671407 2024-03-04T17:50:12.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.968+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=E22823PN4I72 type=del size=2155105 2024-03-04T17:50:12.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.972+0000 7fe960f568c0 1 < completed write operation key=P28N6U79ZUV2 tag=WM5C8J3ZPQY5 type=write size=466806 2024-03-04T17:50:12.978 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.976+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=12Z6M9L83T6L type=del size=1670879 2024-03-04T17:50:12.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.980+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=6XG8G81CAFI9 type=del 2024-03-04T17:50:12.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.984+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=TDSFZLPKT2VL type=del size=106579 2024-03-04T17:50:12.994 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:12.992+0000 7fe960f568c0 1 < suggested remove operation key=3YTAYUQLLUKY tag=BDQOOQN2LKU2 type=del 2024-03-04T17:50:13.002 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.000+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~6X2IPPJFOOJG.0 size=163997 2024-03-04T17:50:13.002 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.000+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=6X2IPPJFOOJG tag=Y58FNJ8S0L5U type=write size=1212573 2024-03-04T17:50:13.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.004+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=NTMQBEU0TZV4 type=del 2024-03-04T17:50:13.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.012+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~TV2FM5BD4RS3.0 size=907782 2024-03-04T17:50:13.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.012+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=TV2FM5BD4RS3 tag=459KXR7EY3H1 type=write size=1956358 2024-03-04T17:50:13.018 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.016+0000 7fe960f568c0 1 < canceled operation key=P28N6U79ZUV2 tag=OF3KW3IZCG26 type=write size=1005912 2024-03-04T17:50:13.022 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.020+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=5Y1P1OL48ATM type=del size=2466376 2024-03-04T17:50:13.026 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.024+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~YFQMSS6G9Q55.0 size=1048576 2024-03-04T17:50:13.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.028+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~YFQMSS6G9Q55.1 size=1048576 2024-03-04T17:50:13.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.032+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~YFQMSS6G9Q55.2 size=808890 2024-03-04T17:50:13.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.036+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=YFQMSS6G9Q55 tag=U0UWONMV3FJ7 type=write size=3954618 2024-03-04T17:50:13.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.040+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=I5H8RQRSH7FJ type=del size=3720651 2024-03-04T17:50:13.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.044+0000 7fe960f568c0 1 < completed write operation key=JSO6WIYMPCF6 tag=X3OGV6QZOPWY type=write size=128031 2024-03-04T17:50:13.053 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.052+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~8BRNQTK8FA9P.0 size=1048576 2024-03-04T17:50:13.056 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.056+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~8BRNQTK8FA9P.1 size=1048576 2024-03-04T17:50:13.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.056+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~8BRNQTK8FA9P.2 size=790837 2024-03-04T17:50:13.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.056+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=8BRNQTK8FA9P tag=BV6IPWG5UBVT type=write size=3936565 2024-03-04T17:50:13.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.060+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~4XD9YX59F1D6.0 size=1048576 2024-03-04T17:50:13.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.060+0000 7fe960f568c0 1 < canceled part key=_multipart_UKZ079SVZZKG.2~4XD9YX59F1D6.1 size=317727 2024-03-04T17:50:13.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.068+0000 7fe960f568c0 1 < canceled multipart upload key=UKZ079SVZZKG upload=4XD9YX59F1D6 tag=S6F0BQFBHE6F type=write size=2414879 2024-03-04T17:50:13.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.072+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=3JWJRE7V5AD9 type=del size=791492 2024-03-04T17:50:13.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.072+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=MTAOXH8QQDEJ type=del 2024-03-04T17:50:13.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.076+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=3T04Y816MORR type=write size=1041975 2024-03-04T17:50:13.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.084+0000 7fe960f568c0 1 < suggested update operation key=2FUTMG0SSXBY tag=ZETGE96F6GQF type=write size=382182 2024-03-04T17:50:13.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.088+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=3M19RW4OV4DW type=del size=4117890 2024-03-04T17:50:13.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.092+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=3FPRBY8HGR0T type=del 2024-03-04T17:50:13.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.096+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=H2ZNL0HI704L type=del size=451953 2024-03-04T17:50:13.101 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.100+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=IQAOCOVWVZVC type=del 2024-03-04T17:50:13.105 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.104+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=SYDHXJYDMDE9 type=del size=623859 2024-03-04T17:50:13.109 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.108+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=NFYMUTGPE48J type=del 2024-03-04T17:50:13.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.116+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~G04JIHCXYV2I.0 size=1048576 2024-03-04T17:50:13.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.116+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~G04JIHCXYV2I.1 size=1048576 2024-03-04T17:50:13.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.120+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~G04JIHCXYV2I.2 size=634102 2024-03-04T17:50:13.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.120+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=G04JIHCXYV2I tag=VOS618A0HIAZ type=write size=3779830 2024-03-04T17:50:13.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.124+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~25LEIKP4TUHT.0 size=1048576 2024-03-04T17:50:13.128 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.128+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~25LEIKP4TUHT.1 size=1048576 2024-03-04T17:50:13.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.128+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~25LEIKP4TUHT.2 size=733278 2024-03-04T17:50:13.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.136+0000 7fe960f568c0 1 < completed multipart upload key=U2G6YEVD5506 upload=25LEIKP4TUHT tag=GCX8F83G3I3T type=write size=3879006 2024-03-04T17:50:13.141 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.140+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=ZNFW338WA5US type=del size=2974410 2024-03-04T17:50:13.150 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.148+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=6AJLXC6X6G4V type=del 2024-03-04T17:50:13.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.152+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=FA5311JVBQXJ type=del size=1038865 2024-03-04T17:50:13.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.156+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=E22823PN4I72 type=del 2024-03-04T17:50:13.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.160+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=RWQXYCNM03X2 type=del size=705472 2024-03-04T17:50:13.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.164+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=12Z6M9L83T6L type=del 2024-03-04T17:50:13.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.172+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~BHAVOEF6QO4R.0 size=1048576 2024-03-04T17:50:13.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.172+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~BHAVOEF6QO4R.1 size=887658 2024-03-04T17:50:13.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.172+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=BHAVOEF6QO4R tag=TRUG13ZPA7JC type=write size=2984810 2024-03-04T17:50:13.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.176+0000 7fe960f568c0 1 < canceled operation key=UKZ079SVZZKG tag=TDSFZLPKT2VL type=del size=106579 2024-03-04T17:50:13.183 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.180+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=WO1CVR9S5I0D type=del size=1405140 2024-03-04T17:50:13.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.184+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~6X2IPPJFOOJG.0 size=163997 2024-03-04T17:50:13.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.192+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=6X2IPPJFOOJG tag=Y58FNJ8S0L5U type=write size=1212573 2024-03-04T17:50:13.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.200+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~QI93V9T3X4GF.0 size=1048576 2024-03-04T17:50:13.203 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.204+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~QI93V9T3X4GF.1 size=692833 2024-03-04T17:50:13.203 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.204+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=QI93V9T3X4GF tag=99JQDCGTTP0B type=write size=2789985 2024-03-04T17:50:13.207 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.208+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~TV2FM5BD4RS3.0 size=907782 2024-03-04T17:50:13.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.212+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=TV2FM5BD4RS3 tag=459KXR7EY3H1 type=write size=1956358 2024-03-04T17:50:13.221 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.220+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~FBBC61HIYSJJ.0 size=1048576 2024-03-04T17:50:13.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.224+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~FBBC61HIYSJJ.1 size=108661 2024-03-04T17:50:13.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.224+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=FBBC61HIYSJJ tag=1J50MN2T9CSC type=write size=2205813 2024-03-04T17:50:13.228 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.228+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=5Y1P1OL48ATM type=del 2024-03-04T17:50:13.232 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.232+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=3MU1CZYJO2GC type=del size=2301137 2024-03-04T17:50:13.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.236+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=I5H8RQRSH7FJ type=del 2024-03-04T17:50:13.240 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.240+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=3HRHAGPKB0LB type=del size=3925457 2024-03-04T17:50:13.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.244+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~8BRNQTK8FA9P.0 size=1048576 2024-03-04T17:50:13.247 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.248+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~8BRNQTK8FA9P.1 size=1048576 2024-03-04T17:50:13.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.248+0000 7fe960f568c0 1 < canceled part key=_multipart_FLYDZHOIWO6L.2~8BRNQTK8FA9P.2 size=790837 2024-03-04T17:50:13.254 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.252+0000 7fe960f568c0 1 < canceled multipart upload key=FLYDZHOIWO6L upload=8BRNQTK8FA9P tag=BV6IPWG5UBVT type=write size=3936565 2024-03-04T17:50:13.258 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.256+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=T6WWRBVGJ67Q type=del size=1661 2024-03-04T17:50:13.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.260+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=3JWJRE7V5AD9 type=del 2024-03-04T17:50:13.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.264+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=4EMAS4XPSC77 type=del size=3913055 2024-03-04T17:50:13.270 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.268+0000 7fe960f568c0 1 < completed write operation key=UEVMIPQ9FR81 tag=3T04Y816MORR type=write size=1041975 2024-03-04T17:50:13.278 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.276+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~WPSU43A0RK3R.0 size=1048576 2024-03-04T17:50:13.280 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.280+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~WPSU43A0RK3R.1 size=779893 2024-03-04T17:50:13.280 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.280+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=WPSU43A0RK3R tag=863YL7HSUY8E type=write size=2877045 2024-03-04T17:50:13.284 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.284+0000 7fe960f568c0 1 < canceled operation key=FLYDZHOIWO6L tag=3M19RW4OV4DW type=del size=4117890 2024-03-04T17:50:13.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.288+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~NR1OEK9NJ7CB.0 size=1048576 2024-03-04T17:50:13.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.292+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~NR1OEK9NJ7CB.1 size=229092 2024-03-04T17:50:13.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.292+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=NR1OEK9NJ7CB tag=JWKZMFO9L72D type=write size=2326244 2024-03-04T17:50:13.300 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.300+0000 7fe960f568c0 1 < suggested remove operation key=XSQ3GE2UHMZG tag=H2ZNL0HI704L type=del 2024-03-04T17:50:13.304 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.304+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=O0SEIN3UI17V type=del size=684158 2024-03-04T17:50:13.311 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.312+0000 7fe960f568c0 1 < suggested update operation key=FSWBX523X9V5 tag=SYDHXJYDMDE9 type=del size=623859 2024-03-04T17:50:13.318 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.316+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~W3WRSPX3J6J0.0 size=1048576 2024-03-04T17:50:13.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.320+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~W3WRSPX3J6J0.1 size=1048576 2024-03-04T17:50:13.324 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.324+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~W3WRSPX3J6J0.2 size=17843 2024-03-04T17:50:13.324 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.324+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=W3WRSPX3J6J0 tag=HD0QI0E2ARYT type=write size=3163571 2024-03-04T17:50:13.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.328+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~G04JIHCXYV2I.0 size=1048576 2024-03-04T17:50:13.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.328+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~G04JIHCXYV2I.1 size=1048576 2024-03-04T17:50:13.334 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.332+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~G04JIHCXYV2I.2 size=634102 2024-03-04T17:50:13.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.340+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=G04JIHCXYV2I tag=VOS618A0HIAZ type=write size=3779830 2024-03-04T17:50:13.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.344+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=0EYH0ZAMFCC7 type=del size=3637193 2024-03-04T17:50:13.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.352+0000 7fe960f568c0 1 < suggested update operation key=NF4OIJWJ83DE tag=ZNFW338WA5US type=del size=2974410 2024-03-04T17:50:13.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.356+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=EFCPNO82QN3U type=write size=1017165 2024-03-04T17:50:13.360 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.360+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=FA5311JVBQXJ type=del 2024-03-04T17:50:13.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.372+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~CI3ERZ062LM9.0 size=1048576 2024-03-04T17:50:13.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.380+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~CI3ERZ062LM9.1 size=15721 2024-03-04T17:50:13.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.380+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=CI3ERZ062LM9 tag=22D450ZRR6XX type=write size=2112873 2024-03-04T17:50:13.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.384+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=RWQXYCNM03X2 type=del 2024-03-04T17:50:13.390 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.388+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~QDYMU0TRNW8G.0 size=147340 2024-03-04T17:50:13.390 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.388+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=QDYMU0TRNW8G tag=EEBXTCM35AEJ type=write size=1195916 2024-03-04T17:50:13.393 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.392+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~BHAVOEF6QO4R.0 size=1048576 2024-03-04T17:50:13.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.396+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~BHAVOEF6QO4R.1 size=887658 2024-03-04T17:50:13.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.400+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=BHAVOEF6QO4R tag=TRUG13ZPA7JC type=write size=2984810 2024-03-04T17:50:13.405 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.404+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=EAAJKDR46FUR type=write size=802792 2024-03-04T17:50:13.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.408+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=WO1CVR9S5I0D type=del 2024-03-04T17:50:13.414 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.412+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=EPQ3NB73FA7F type=del size=972115 2024-03-04T17:50:13.417 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.416+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~QI93V9T3X4GF.0 size=1048576 2024-03-04T17:50:13.420 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.420+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~QI93V9T3X4GF.1 size=692833 2024-03-04T17:50:13.425 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.424+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=QI93V9T3X4GF tag=99JQDCGTTP0B type=write size=2789985 2024-03-04T17:50:13.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.428+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=N37BP5C0I4JB type=del size=3647423 2024-03-04T17:50:13.430 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.428+0000 7fe960f568c0 1 < canceled part key=_multipart_UEVMIPQ9FR81.2~FBBC61HIYSJJ.0 size=1048576 2024-03-04T17:50:13.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.432+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~FBBC61HIYSJJ.1 size=108661 2024-03-04T17:50:13.442 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.440+0000 7fe960f568c0 1 < canceled multipart upload key=UEVMIPQ9FR81 upload=FBBC61HIYSJJ tag=1J50MN2T9CSC type=write size=2205813 2024-03-04T17:50:13.448 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.448+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~MJ7487RV7TNV.0 size=1048576 2024-03-04T17:50:13.451 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.452+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~MJ7487RV7TNV.1 size=1048576 2024-03-04T17:50:13.454 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.452+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~MJ7487RV7TNV.2 size=408323 2024-03-04T17:50:13.454 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.452+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=MJ7487RV7TNV tag=J769VFY0BURD type=write size=3554051 2024-03-04T17:50:13.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.456+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=3MU1CZYJO2GC type=del 2024-03-04T17:50:13.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.464+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~10LPMV8MOEU2.0 size=253511 2024-03-04T17:50:13.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.464+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=10LPMV8MOEU2 tag=K8F51AG5QL2U type=write size=1302087 2024-03-04T17:50:13.468 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.468+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=3HRHAGPKB0LB type=del 2024-03-04T17:50:13.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.468+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=0ZORMLZT924H type=del size=891183 2024-03-04T17:50:13.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.472+0000 7fe960f568c0 -1 < failed to complete operation key=NF4OIJWJ83DE tag=T6WWRBVGJ67Q type=del size=1661: (22) Invalid argument 2024-03-04T17:50:13.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.480+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~ZGDA91V1K21M.0 size=1048576 2024-03-04T17:50:13.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.480+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~ZGDA91V1K21M.1 size=1048576 2024-03-04T17:50:13.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.484+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~ZGDA91V1K21M.2 size=259159 2024-03-04T17:50:13.486 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.484+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=ZGDA91V1K21M tag=Y32Q1RLH5T54 type=write size=3404887 2024-03-04T17:50:13.489 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.488+0000 7fe960f568c0 1 < canceled operation key=GZ4RRIR5CC26 tag=4EMAS4XPSC77 type=del size=3913055 2024-03-04T17:50:13.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.492+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=QR6WAI236VTX type=del size=4188696 2024-03-04T17:50:13.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.496+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~WPSU43A0RK3R.0 size=1048576 2024-03-04T17:50:13.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.496+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~WPSU43A0RK3R.1 size=779893 2024-03-04T17:50:13.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.504+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=WPSU43A0RK3R tag=863YL7HSUY8E type=write size=2877045 2024-03-04T17:50:13.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.512+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~T1598OZ73GE2.0 size=1048576 2024-03-04T17:50:13.514 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.512+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~T1598OZ73GE2.1 size=1048576 2024-03-04T17:50:13.516 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.516+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~T1598OZ73GE2.2 size=354713 2024-03-04T17:50:13.516 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.516+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=T1598OZ73GE2 tag=63GFT1SRFDDP type=write size=3500441 2024-03-04T17:50:13.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.520+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~NR1OEK9NJ7CB.0 size=1048576 2024-03-04T17:50:13.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.520+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~NR1OEK9NJ7CB.1 size=229092 2024-03-04T17:50:13.527 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.524+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=NR1OEK9NJ7CB tag=JWKZMFO9L72D type=write size=2326244 2024-03-04T17:50:13.530 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.528+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=QWGWTI1HBZBS type=write size=187758 2024-03-04T17:50:13.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.532+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=O0SEIN3UI17V type=del 2024-03-04T17:50:13.536 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.536+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=IXF4DXD6XUGX type=write size=72430 2024-03-04T17:50:13.540 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.540+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~W3WRSPX3J6J0.0 size=1048576 2024-03-04T17:50:13.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.540+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~W3WRSPX3J6J0.1 size=1048576 2024-03-04T17:50:13.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.544+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~W3WRSPX3J6J0.2 size=17843 2024-03-04T17:50:13.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.548+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=W3WRSPX3J6J0 tag=HD0QI0E2ARYT type=write size=3163571 2024-03-04T17:50:13.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.556+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~O3A2V2O8DF6H.0 size=1048576 2024-03-04T17:50:13.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.560+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~O3A2V2O8DF6H.1 size=269236 2024-03-04T17:50:13.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.560+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=O3A2V2O8DF6H tag=4975OI7M1CNM type=write size=2366388 2024-03-04T17:50:13.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.560+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=0EYH0ZAMFCC7 type=del 2024-03-04T17:50:13.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.564+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=HKSH6UZQMEK3 type=del size=872866 2024-03-04T17:50:13.570 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.568+0000 7fe960f568c0 1 < completed write operation key=GZ4RRIR5CC26 tag=EFCPNO82QN3U type=write size=1017165 2024-03-04T17:50:13.573 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.572+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=UTB9CM9D5402 type=del size=137199 2024-03-04T17:50:13.577 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.576+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~CI3ERZ062LM9.0 size=1048576 2024-03-04T17:50:13.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.576+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~CI3ERZ062LM9.1 size=15721 2024-03-04T17:50:13.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.584+0000 7fe960f568c0 1 < completed multipart upload key=2MKQA5JYWU8L upload=CI3ERZ062LM9 tag=22D450ZRR6XX type=write size=2112873 2024-03-04T17:50:13.588 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.588+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=X4L0ZXT3ARL2 type=write size=106948 2024-03-04T17:50:13.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.592+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~QDYMU0TRNW8G.0 size=147340 2024-03-04T17:50:13.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.596+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=QDYMU0TRNW8G tag=EEBXTCM35AEJ type=write size=1195916 2024-03-04T17:50:13.601 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.600+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=M6XDTJHPB0CM type=del size=1971485 2024-03-04T17:50:13.604 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.604+0000 7fe960f568c0 1 < completed write operation key=GZ4RRIR5CC26 tag=EAAJKDR46FUR type=write size=802792 2024-03-04T17:50:13.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.608+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=3T5XI3M5POH4 type=del size=600099 2024-03-04T17:50:13.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.612+0000 7fe960f568c0 1 < completed delete operation key=ASC735ZCBLCG tag=EPQ3NB73FA7F type=del 2024-03-04T17:50:13.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.616+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~I88NHH6NH7CK.0 size=279439 2024-03-04T17:50:13.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.616+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=I88NHH6NH7CK tag=EWB98CA3UQ14 type=write size=1328015 2024-03-04T17:50:13.623 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.620+0000 7fe960f568c0 1 < canceled operation key=UEVMIPQ9FR81 tag=N37BP5C0I4JB type=del size=3647423 2024-03-04T17:50:13.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.624+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=F87WKHKA5U3P type=del size=1919592 2024-03-04T17:50:13.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.632+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~MJ7487RV7TNV.0 size=1048576 2024-03-04T17:50:13.635 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.632+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~MJ7487RV7TNV.1 size=1048576 2024-03-04T17:50:13.638 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.636+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~MJ7487RV7TNV.2 size=408323 2024-03-04T17:50:13.644 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.644+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=MJ7487RV7TNV tag=J769VFY0BURD type=write size=3554051 2024-03-04T17:50:13.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.648+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~0BUZWTX3W2J6.0 size=353729 2024-03-04T17:50:13.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.648+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=0BUZWTX3W2J6 tag=TW3YK9VZ3P39 type=write size=1402305 2024-03-04T17:50:13.655 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.652+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~10LPMV8MOEU2.0 size=253511 2024-03-04T17:50:13.661 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.660+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=10LPMV8MOEU2 tag=K8F51AG5QL2U type=write size=1302087 2024-03-04T17:50:13.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.664+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=560L1LVHR65T type=del size=2113643 2024-03-04T17:50:13.668 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.668+0000 7fe960f568c0 1 < canceled operation key=6X4ZV3BFZNDL tag=0ZORMLZT924H type=del size=891183 2024-03-04T17:50:13.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.672+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=Q63NWWSM6BS5 type=write size=92941 2024-03-04T17:50:13.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.676+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~ZGDA91V1K21M.0 size=1048576 2024-03-04T17:50:13.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.680+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~ZGDA91V1K21M.1 size=1048576 2024-03-04T17:50:13.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.680+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~ZGDA91V1K21M.2 size=259159 2024-03-04T17:50:13.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.688+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=ZGDA91V1K21M tag=Y32Q1RLH5T54 type=write size=3404887 2024-03-04T17:50:13.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.692+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~5M9JCU32ZRVO.0 size=233335 2024-03-04T17:50:13.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.692+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=5M9JCU32ZRVO tag=EZR79VFQH4JX type=write size=1281911 2024-03-04T17:50:13.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.696+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=QR6WAI236VTX type=del 2024-03-04T17:50:13.703 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.700+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=YGT9ESIOYGZ6 type=del size=1325118 2024-03-04T17:50:13.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.704+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~T1598OZ73GE2.0 size=1048576 2024-03-04T17:50:13.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.708+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~T1598OZ73GE2.1 size=1048576 2024-03-04T17:50:13.712 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.712+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~T1598OZ73GE2.2 size=354713 2024-03-04T17:50:13.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.716+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=T1598OZ73GE2 tag=63GFT1SRFDDP type=write size=3500441 2024-03-04T17:50:13.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.720+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=VOP22B74RHV8 type=del size=2873345 2024-03-04T17:50:13.730 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.728+0000 7fe960f568c0 1 < suggested remove operation key=XSQ3GE2UHMZG tag=QWGWTI1HBZBS type=write 2024-03-04T17:50:13.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.732+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=8YU9HX0YMOTG type=del size=1727965 2024-03-04T17:50:13.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.736+0000 7fe960f568c0 1 < completed write operation key=78N8HHC4O4CG tag=IXF4DXD6XUGX type=write size=72430 2024-03-04T17:50:13.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.740+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=251IU1GKR4JJ type=del size=293278 2024-03-04T17:50:13.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.744+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~O3A2V2O8DF6H.0 size=1048576 2024-03-04T17:50:13.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.748+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~O3A2V2O8DF6H.1 size=269236 2024-03-04T17:50:13.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.752+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=O3A2V2O8DF6H tag=4975OI7M1CNM type=write size=2366388 2024-03-04T17:50:13.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.756+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=GSZYR45CUWCY type=del size=130604 2024-03-04T17:50:13.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.760+0000 7fe960f568c0 1 < completed delete operation key=ASC735ZCBLCG tag=HKSH6UZQMEK3 type=del 2024-03-04T17:50:13.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.768+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~KCO7KDEGCILO.0 size=219856 2024-03-04T17:50:13.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.768+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=KCO7KDEGCILO tag=6O0QGCLLK931 type=write size=1268432 2024-03-04T17:50:13.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.772+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=UTB9CM9D5402 type=del 2024-03-04T17:50:13.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.772+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=S2TLC3F4G49T type=del size=1277839 2024-03-04T17:50:13.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.776+0000 7fe960f568c0 1 < completed write operation key=768X2XKWJW1S tag=X4L0ZXT3ARL2 type=write size=106948 2024-03-04T17:50:13.785 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.784+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~2Q04FVLMGN55.0 size=1048576 2024-03-04T17:50:13.788 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.788+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~2Q04FVLMGN55.1 size=1048576 2024-03-04T17:50:13.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.788+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~2Q04FVLMGN55.2 size=12574 2024-03-04T17:50:13.791 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.788+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=2Q04FVLMGN55 tag=FUM9W9WSXGZF type=write size=3158302 2024-03-04T17:50:13.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.792+0000 7fe960f568c0 1 < completed delete operation key=ASC735ZCBLCG tag=M6XDTJHPB0CM type=del 2024-03-04T17:50:13.798 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.796+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=IGW5TLOL6QYJ type=del size=4025297 2024-03-04T17:50:13.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.804+0000 7fe960f568c0 1 < suggested remove operation key=BFF9OD00TU54 tag=3T5XI3M5POH4 type=del 2024-03-04T17:50:13.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.812+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~NM5KKTTZ15WT.0 size=102280 2024-03-04T17:50:13.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.812+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=NM5KKTTZ15WT tag=V02Q4NV4CKNS type=write size=1150856 2024-03-04T17:50:13.816 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.816+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~I88NHH6NH7CK.0 size=279439 2024-03-04T17:50:13.822 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.820+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=I88NHH6NH7CK tag=EWB98CA3UQ14 type=write size=1328015 2024-03-04T17:50:13.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.824+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=I93FACRDQP9Y type=del size=1473484 2024-03-04T17:50:13.830 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.828+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=F87WKHKA5U3P type=del 2024-03-04T17:50:13.834 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.832+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=UXATL0WN5QF5 type=del size=3335725 2024-03-04T17:50:13.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.840+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~0BUZWTX3W2J6.0 size=353729 2024-03-04T17:50:13.847 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.844+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=0BUZWTX3W2J6 tag=TW3YK9VZ3P39 type=write size=1402305 2024-03-04T17:50:13.851 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.848+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=S2269B37Z1HY type=write size=429643 2024-03-04T17:50:13.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.856+0000 7fe960f568c0 1 < suggested remove operation key=UEVMIPQ9FR81 tag=560L1LVHR65T type=del 2024-03-04T17:50:13.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.860+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=SCFPTLQYP00I type=write size=78175 2024-03-04T17:50:13.866 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.864+0000 7fe960f568c0 1 < completed write operation key=QFF67JIW8Y8E tag=Q63NWWSM6BS5 type=write size=92941 2024-03-04T17:50:13.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.868+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=HLKIYEOCD6B4 type=del size=32307 2024-03-04T17:50:13.874 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.872+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~5M9JCU32ZRVO.0 size=233335 2024-03-04T17:50:13.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.880+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=5M9JCU32ZRVO tag=EZR79VFQH4JX type=write size=1281911 2024-03-04T17:50:13.887 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.884+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~XR41H32VURQP.0 size=1048576 2024-03-04T17:50:13.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.888+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~XR41H32VURQP.1 size=922488 2024-03-04T17:50:13.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.888+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=XR41H32VURQP tag=MHV564EIOFC2 type=write size=3019640 2024-03-04T17:50:13.894 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.892+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=YGT9ESIOYGZ6 type=del 2024-03-04T17:50:13.898 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.896+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=ATX1J9E0UU88 type=del size=222458 2024-03-04T17:50:13.902 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.900+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=VOP22B74RHV8 type=del 2024-03-04T17:50:13.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.904+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=GYCIG0537LAY type=write size=809603 2024-03-04T17:50:13.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.908+0000 7fe960f568c0 -1 < failed to complete operation key=BFF9OD00TU54 tag=8YU9HX0YMOTG type=del size=1727965: (22) Invalid argument 2024-03-04T17:50:13.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.916+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~YGCYNQ7EBYX9.0 size=1048576 2024-03-04T17:50:13.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.920+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~YGCYNQ7EBYX9.1 size=77264 2024-03-04T17:50:13.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.920+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=YGCYNQ7EBYX9 tag=D43RJSBW2MFQ type=write size=2174416 2024-03-04T17:50:13.925 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.924+0000 7fe960f568c0 1 < canceled operation key=WNXVI6K5MU3R tag=251IU1GKR4JJ type=del size=293278 2024-03-04T17:50:13.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.928+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=4X4ZO7NTJKT8 type=del size=741829 2024-03-04T17:50:13.933 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.932+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=GSZYR45CUWCY type=del 2024-03-04T17:50:13.938 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.936+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=TGPHD62ZKZZM type=del size=1186033 2024-03-04T17:50:13.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.940+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~KCO7KDEGCILO.0 size=219856 2024-03-04T17:50:13.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.948+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=KCO7KDEGCILO tag=6O0QGCLLK931 type=write size=1268432 2024-03-04T17:50:13.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.952+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~MZ5I8E8365RW.0 size=1048576 2024-03-04T17:50:13.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.956+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~MZ5I8E8365RW.1 size=207686 2024-03-04T17:50:13.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.956+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=MZ5I8E8365RW tag=9J12T2YL29UP type=write size=2304838 2024-03-04T17:50:13.962 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.960+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=S2TLC3F4G49T type=del 2024-03-04T17:50:13.969 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.968+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~2ER28F2BNW7B.0 size=195338 2024-03-04T17:50:13.969 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.968+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=2ER28F2BNW7B tag=HWYWNIK6IY50 type=write size=1243914 2024-03-04T17:50:13.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.972+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~2Q04FVLMGN55.0 size=1048576 2024-03-04T17:50:13.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.976+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~2Q04FVLMGN55.1 size=1048576 2024-03-04T17:50:13.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.976+0000 7fe960f568c0 1 < canceled part key=_multipart_QGLN27WH1N2O.2~2Q04FVLMGN55.2 size=12574 2024-03-04T17:50:13.983 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.980+0000 7fe960f568c0 1 < canceled multipart upload key=QGLN27WH1N2O upload=2Q04FVLMGN55 tag=FUM9W9WSXGZF type=write size=3158302 2024-03-04T17:50:13.990 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.988+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~5PYBDSZIVIKG.0 size=706507 2024-03-04T17:50:13.990 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.988+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=5PYBDSZIVIKG tag=1HLR08KSGKGW type=write size=1755083 2024-03-04T17:50:13.993 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.992+0000 7fe960f568c0 1 < canceled operation key=QGLN27WH1N2O tag=IGW5TLOL6QYJ type=del size=4025297 2024-03-04T17:50:13.997 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.996+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=TBE67KYQJP0F type=write size=1015192 2024-03-04T17:50:13.998 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:13.996+0000 7fe960f568c0 1 < canceled part key=_multipart_QGLN27WH1N2O.2~NM5KKTTZ15WT.0 size=102280 2024-03-04T17:50:14.004 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.004+0000 7fe960f568c0 1 < canceled multipart upload key=QGLN27WH1N2O upload=NM5KKTTZ15WT tag=V02Q4NV4CKNS type=write size=1150856 2024-03-04T17:50:14.007 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.008+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=P6GU2ZYUA338 type=write size=230310 2024-03-04T17:50:14.011 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.012+0000 7fe960f568c0 1 < canceled operation key=WNXVI6K5MU3R tag=I93FACRDQP9Y type=del size=1473484 2024-03-04T17:50:14.015 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.016+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=D2B8ICN3OEIC type=del size=2860463 2024-03-04T17:50:14.019 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.016+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=UXATL0WN5QF5 type=del 2024-03-04T17:50:14.026 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.024+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~FCPC2IFM4OJR.0 size=343661 2024-03-04T17:50:14.026 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.024+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=FCPC2IFM4OJR tag=3VLF155VVD97 type=write size=1392237 2024-03-04T17:50:14.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.028+0000 7fe960f568c0 1 < completed write operation key=ASC735ZCBLCG tag=S2269B37Z1HY type=write size=429643 2024-03-04T17:50:14.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.036+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~G9T2G10IKNS9.0 size=138018 2024-03-04T17:50:14.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.036+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=G9T2G10IKNS9 tag=C7QUVJ7SJATB type=write size=1186594 2024-03-04T17:50:14.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.040+0000 7fe960f568c0 1 < completed write operation key=2FUTMG0SSXBY tag=SCFPTLQYP00I type=write size=78175 2024-03-04T17:50:14.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.044+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~WI19QX8OTKMG.0 size=1048576 2024-03-04T17:50:14.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.048+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~WI19QX8OTKMG.1 size=115582 2024-03-04T17:50:14.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.048+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=WI19QX8OTKMG tag=C7XV0IW4BSIK type=write size=2212734 2024-03-04T17:50:14.053 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.052+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=HLKIYEOCD6B4 type=del 2024-03-04T17:50:14.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.056+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=UJ5FKQKJZCWJ type=del size=2159334 2024-03-04T17:50:14.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.060+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~XR41H32VURQP.0 size=1048576 2024-03-04T17:50:14.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.064+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~XR41H32VURQP.1 size=922488 2024-03-04T17:50:14.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.068+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=XR41H32VURQP tag=MHV564EIOFC2 type=write size=3019640 2024-03-04T17:50:14.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.072+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=CL8F0IJGKMOR type=del size=2951581 2024-03-04T17:50:14.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.080+0000 7fe960f568c0 1 < suggested update operation key=UEVMIPQ9FR81 tag=ATX1J9E0UU88 type=del size=222458 2024-03-04T17:50:14.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.084+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=D843JF8GCGB5 type=del size=3080801 2024-03-04T17:50:14.091 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.088+0000 7fe960f568c0 1 < completed write operation key=JJ7SM1Y73A4C tag=GYCIG0537LAY type=write size=809603 2024-03-04T17:50:14.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.092+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=NN8H69DHAPQK type=del size=131816 2024-03-04T17:50:14.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.096+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~YGCYNQ7EBYX9.0 size=1048576 2024-03-04T17:50:14.101 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.100+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~YGCYNQ7EBYX9.1 size=77264 2024-03-04T17:50:14.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.108+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=YGCYNQ7EBYX9 tag=D43RJSBW2MFQ type=write size=2174416 2024-03-04T17:50:14.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.112+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=4WHB06RO1ZXA type=del size=3992956 2024-03-04T17:50:14.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.116+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=4X4ZO7NTJKT8 type=del 2024-03-04T17:50:14.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.116+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=AQMTR4PN935I type=del size=1159535 2024-03-04T17:50:14.127 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.124+0000 7fe960f568c0 1 < suggested remove operation key=6X4ZV3BFZNDL tag=TGPHD62ZKZZM type=del 2024-03-04T17:50:14.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.128+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=DRCUTPRIZ5ST type=del size=1651239 2024-03-04T17:50:14.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.132+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~MZ5I8E8365RW.0 size=1048576 2024-03-04T17:50:14.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.136+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~MZ5I8E8365RW.1 size=207686 2024-03-04T17:50:14.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.144+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=MZ5I8E8365RW tag=9J12T2YL29UP type=write size=2304838 2024-03-04T17:50:14.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.148+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=4DTQC22P3LKG type=del size=4148929 2024-03-04T17:50:14.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.152+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~2ER28F2BNW7B.0 size=195338 2024-03-04T17:50:14.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.156+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=2ER28F2BNW7B tag=HWYWNIK6IY50 type=write size=1243914 2024-03-04T17:50:14.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.164+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~4K8RUSSGCQL5.0 size=1048576 2024-03-04T17:50:14.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.168+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~4K8RUSSGCQL5.1 size=1048576 2024-03-04T17:50:14.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.168+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~4K8RUSSGCQL5.2 size=773876 2024-03-04T17:50:14.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.168+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=4K8RUSSGCQL5 tag=RVGDJ9U6PCAC type=write size=3919604 2024-03-04T17:50:14.174 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.172+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~5PYBDSZIVIKG.0 size=706507 2024-03-04T17:50:14.180 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.180+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=5PYBDSZIVIKG tag=1HLR08KSGKGW type=write size=1755083 2024-03-04T17:50:14.184 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.184+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=BRJFVD5TSN8S type=write size=273029 2024-03-04T17:50:14.188 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.188+0000 7fe960f568c0 1 < completed write operation key=2MKQA5JYWU8L tag=TBE67KYQJP0F type=write size=1015192 2024-03-04T17:50:14.191 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.188+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=UZ4ERR46Y2N1 type=del size=3941783 2024-03-04T17:50:14.201 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.200+0000 7fe960f568c0 1 < suggested remove operation key=7Z7BPZ8XGBHZ tag=P6GU2ZYUA338 type=write 2024-03-04T17:50:14.204 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.204+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=B3RVLVSFMA3J type=del size=3523623 2024-03-04T17:50:14.208 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.208+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=D2B8ICN3OEIC type=del 2024-03-04T17:50:14.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.212+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=7607F1Z709F5 type=write size=268772 2024-03-04T17:50:14.216 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.216+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~FCPC2IFM4OJR.0 size=343661 2024-03-04T17:50:14.222 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.220+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=FCPC2IFM4OJR tag=3VLF155VVD97 type=write size=1392237 2024-03-04T17:50:14.229 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.228+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~6KCWTE9X361Z.0 size=808546 2024-03-04T17:50:14.229 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.228+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=6KCWTE9X361Z tag=RY44XMJ9F2JT type=write size=1857122 2024-03-04T17:50:14.233 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.232+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~G9T2G10IKNS9.0 size=138018 2024-03-04T17:50:14.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.236+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=G9T2G10IKNS9 tag=C7QUVJ7SJATB type=write size=1186594 2024-03-04T17:50:14.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.240+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=ODOF0WDDUZ1B type=write size=468142 2024-03-04T17:50:14.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.244+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~WI19QX8OTKMG.0 size=1048576 2024-03-04T17:50:14.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.248+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~WI19QX8OTKMG.1 size=115582 2024-03-04T17:50:14.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.252+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=WI19QX8OTKMG tag=C7XV0IW4BSIK type=write size=2212734 2024-03-04T17:50:14.259 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.256+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=MKYPBD0LIQAI type=del size=3839670 2024-03-04T17:50:14.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.260+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=UJ5FKQKJZCWJ type=del 2024-03-04T17:50:14.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.264+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=8HG1FN4XPAVL type=del size=1233691 2024-03-04T17:50:14.270 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.268+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=CL8F0IJGKMOR type=del 2024-03-04T17:50:14.277 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.276+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~4I80DDXM2UVL.0 size=164307 2024-03-04T17:50:14.277 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.276+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=4I80DDXM2UVL tag=PY2RQF9H8REK type=write size=1212883 2024-03-04T17:50:14.281 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.280+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=D843JF8GCGB5 type=del 2024-03-04T17:50:14.285 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.284+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=5JW7GF9MT9J7 type=del size=2271210 2024-03-04T17:50:14.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.292+0000 7fe960f568c0 1 < suggested update operation key=FLYDZHOIWO6L tag=NN8H69DHAPQK type=del size=131816 2024-03-04T17:50:14.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.300+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~83BFD6TLDV8Y.0 size=1048576 2024-03-04T17:50:14.302 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.300+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~83BFD6TLDV8Y.1 size=1048576 2024-03-04T17:50:14.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.304+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~83BFD6TLDV8Y.2 size=840337 2024-03-04T17:50:14.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.304+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=83BFD6TLDV8Y tag=OXMAY17MCJOA type=write size=3986065 2024-03-04T17:50:14.310 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.308+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=4WHB06RO1ZXA type=del 2024-03-04T17:50:14.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.316+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~N7PUF3VRQTYA.0 size=1048576 2024-03-04T17:50:14.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.320+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~N7PUF3VRQTYA.1 size=1048576 2024-03-04T17:50:14.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.320+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~N7PUF3VRQTYA.2 size=462920 2024-03-04T17:50:14.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.320+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=N7PUF3VRQTYA tag=WBQG6UU928GE type=write size=3608648 2024-03-04T17:50:14.326 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.324+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=AQMTR4PN935I type=del 2024-03-04T17:50:14.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.332+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~HTJGJUZOSFDT.0 size=1048576 2024-03-04T17:50:14.336 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.336+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~HTJGJUZOSFDT.1 size=1048576 2024-03-04T17:50:14.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.336+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~HTJGJUZOSFDT.2 size=559701 2024-03-04T17:50:14.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.336+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=HTJGJUZOSFDT tag=ZPPY0S9MKKJG type=write size=3705429 2024-03-04T17:50:14.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.340+0000 7fe960f568c0 1 < canceled operation key=8PZD01OOX3AF tag=DRCUTPRIZ5ST type=del size=1651239 2024-03-04T17:50:14.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.344+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=B5RUPSJW2HCR type=del size=1130914 2024-03-04T17:50:14.350 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.348+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=4DTQC22P3LKG type=del 2024-03-04T17:50:14.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.352+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=3S3NVTCAYN2F type=del size=2753109 2024-03-04T17:50:14.358 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.356+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~4K8RUSSGCQL5.0 size=1048576 2024-03-04T17:50:14.361 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.360+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~4K8RUSSGCQL5.1 size=1048576 2024-03-04T17:50:14.364 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.364+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~4K8RUSSGCQL5.2 size=773876 2024-03-04T17:50:14.370 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.368+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=4K8RUSSGCQL5 tag=RVGDJ9U6PCAC type=write size=3919604 2024-03-04T17:50:14.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.372+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=F712RK4L7MM8 type=del size=3843198 2024-03-04T17:50:14.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.376+0000 7fe960f568c0 1 < canceled operation key=JSO6WIYMPCF6 tag=BRJFVD5TSN8S type=write size=273029 2024-03-04T17:50:14.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.380+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=IYAQI1DZC8NA type=del size=1837347 2024-03-04T17:50:14.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.384+0000 7fe960f568c0 -1 < failed to complete operation key=7Z7BPZ8XGBHZ tag=UZ4ERR46Y2N1 type=del size=3941783: (22) Invalid argument 2024-03-04T17:50:14.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.388+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=8OUL6XAA33GC type=del size=2032441 2024-03-04T17:50:14.396 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.396+0000 7fe960f568c0 1 < suggested remove operation key=P28N6U79ZUV2 tag=B3RVLVSFMA3J type=del 2024-03-04T17:50:14.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.400+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=DV8G84D96631 type=del size=2985694 2024-03-04T17:50:14.403 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.404+0000 7fe960f568c0 1 < completed write operation key=2KZ4IFW54C5B tag=7607F1Z709F5 type=write size=268772 2024-03-04T17:50:14.410 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.408+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~XRQFXIDLWM09.0 size=929245 2024-03-04T17:50:14.410 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.408+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=XRQFXIDLWM09 tag=ULVY3YCOO9S3 type=write size=1977821 2024-03-04T17:50:14.414 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.412+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~6KCWTE9X361Z.0 size=808546 2024-03-04T17:50:14.420 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.420+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=6KCWTE9X361Z tag=RY44XMJ9F2JT type=write size=1857122 2024-03-04T17:50:14.424 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.424+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=U65R2MGV7PK1 type=del size=1923510 2024-03-04T17:50:14.428 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.428+0000 7fe960f568c0 1 < completed write operation key=GZ4RRIR5CC26 tag=ODOF0WDDUZ1B type=write size=468142 2024-03-04T17:50:14.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.432+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=93BUOU1LSMXA type=write size=601035 2024-03-04T17:50:14.436 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.436+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=MKYPBD0LIQAI type=del 2024-03-04T17:50:14.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.440+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=QQOD6MQCUDM7 type=del size=3813630 2024-03-04T17:50:14.444 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.444+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=8HG1FN4XPAVL type=del 2024-03-04T17:50:14.448 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.448+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=TN823R7QY7T4 type=write size=341090 2024-03-04T17:50:14.452 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.452+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~4I80DDXM2UVL.0 size=164307 2024-03-04T17:50:14.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.456+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=4I80DDXM2UVL tag=PY2RQF9H8REK type=write size=1212883 2024-03-04T17:50:14.462 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.460+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=LH8T2PJF54EU type=write size=715730 2024-03-04T17:50:14.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.464+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=5JW7GF9MT9J7 type=del 2024-03-04T17:50:14.474 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.472+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~4BAEJGO5EWEU.0 size=1048576 2024-03-04T17:50:14.476 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.476+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~4BAEJGO5EWEU.1 size=1048576 2024-03-04T17:50:14.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.476+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~4BAEJGO5EWEU.2 size=211062 2024-03-04T17:50:14.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.476+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=4BAEJGO5EWEU tag=EGU8DI15PKIL type=write size=3356790 2024-03-04T17:50:14.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.484+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~83BFD6TLDV8Y.0 size=1048576 2024-03-04T17:50:14.486 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.484+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~83BFD6TLDV8Y.1 size=1048576 2024-03-04T17:50:14.486 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.484+0000 7fe960f568c0 1 < canceled part key=_multipart_FLYDZHOIWO6L.2~83BFD6TLDV8Y.2 size=840337 2024-03-04T17:50:14.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.492+0000 7fe960f568c0 1 < canceled multipart upload key=FLYDZHOIWO6L upload=83BFD6TLDV8Y tag=OXMAY17MCJOA type=write size=3986065 2024-03-04T17:50:14.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.496+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=XKMLCQANY3JE type=del size=2796696 2024-03-04T17:50:14.500 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.500+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~N7PUF3VRQTYA.0 size=1048576 2024-03-04T17:50:14.500 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.500+0000 7fe960f568c0 1 < canceled part key=_multipart_BFF9OD00TU54.2~N7PUF3VRQTYA.1 size=1048576 2024-03-04T17:50:14.503 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.500+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~N7PUF3VRQTYA.2 size=462920 2024-03-04T17:50:14.509 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.508+0000 7fe960f568c0 1 < canceled multipart upload key=BFF9OD00TU54 upload=N7PUF3VRQTYA tag=WBQG6UU928GE type=write size=3608648 2024-03-04T17:50:14.516 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.516+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~OBI7V6LB2IMC.0 size=650500 2024-03-04T17:50:14.516 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.516+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=OBI7V6LB2IMC tag=YF6HUCQUC0AH type=write size=1699076 2024-03-04T17:50:14.517 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.516+0000 7fe960f568c0 1 < canceled part key=_multipart_ASC735ZCBLCG.2~HTJGJUZOSFDT.0 size=1048576 2024-03-04T17:50:14.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.520+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~HTJGJUZOSFDT.1 size=1048576 2024-03-04T17:50:14.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.524+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~HTJGJUZOSFDT.2 size=559701 2024-03-04T17:50:14.530 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.528+0000 7fe960f568c0 1 < canceled multipart upload key=ASC735ZCBLCG upload=HTJGJUZOSFDT tag=ZPPY0S9MKKJG type=write size=3705429 2024-03-04T17:50:14.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.532+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=KI86Q66HPDBW type=del size=2709416 2024-03-04T17:50:14.537 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.536+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=B5RUPSJW2HCR type=del 2024-03-04T17:50:14.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.540+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=GA1XDC7GQRBZ type=del size=2418881 2024-03-04T17:50:14.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.544+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=3S3NVTCAYN2F type=del 2024-03-04T17:50:14.554 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.552+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~4DU7J0VS7PYX.0 size=1048576 2024-03-04T17:50:14.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.556+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~4DU7J0VS7PYX.1 size=1048576 2024-03-04T17:50:14.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.560+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~4DU7J0VS7PYX.2 size=361450 2024-03-04T17:50:14.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.560+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=4DU7J0VS7PYX tag=8RP63HX6XXPO type=write size=3507178 2024-03-04T17:50:14.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.564+0000 7fe960f568c0 1 < suggested update operation key=7Z7BPZ8XGBHZ tag=F712RK4L7MM8 type=del size=3843198 2024-03-04T17:50:14.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.568+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=NUJ579B9482Q type=del size=1974279 2024-03-04T17:50:14.574 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.572+0000 7fe960f568c0 -1 < failed to complete operation key=P28N6U79ZUV2 tag=IYAQI1DZC8NA type=del size=1837347: (22) Invalid argument 2024-03-04T17:50:14.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.576+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=VBZO3H5GPINM type=write size=175041 2024-03-04T17:50:14.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.580+0000 7fe960f568c0 1 < canceled operation key=NF4OIJWJ83DE tag=8OUL6XAA33GC type=del size=2032441 2024-03-04T17:50:14.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.584+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=NPWTZR6VRKXW type=del size=4172288 2024-03-04T17:50:14.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.592+0000 7fe960f568c0 1 < suggested update operation key=GZ4RRIR5CC26 tag=DV8G84D96631 type=del size=2985694 2024-03-04T17:50:14.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.596+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=28JV80VENGL3 type=del size=2396777 2024-03-04T17:50:14.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.600+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~XRQFXIDLWM09.0 size=929245 2024-03-04T17:50:14.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.608+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=XRQFXIDLWM09 tag=ULVY3YCOO9S3 type=write size=1977821 2024-03-04T17:50:14.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.612+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=1N3QME40GO0B type=del size=1389749 2024-03-04T17:50:14.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.616+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=U65R2MGV7PK1 type=del 2024-03-04T17:50:14.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.620+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=6XIV2IB3323W type=del size=2905753 2024-03-04T17:50:14.623 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.624+0000 7fe960f568c0 1 < completed write operation key=JSO6WIYMPCF6 tag=93BUOU1LSMXA type=write size=601035 2024-03-04T17:50:14.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.624+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=T933H4UX6QYK type=write size=929927 2024-03-04T17:50:14.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.628+0000 7fe960f568c0 1 < canceled operation key=JT02WL9JYTNM tag=QQOD6MQCUDM7 type=del size=3813630 2024-03-04T17:50:14.638 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.636+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~LZW6IEE3HT8M.0 size=1048576 2024-03-04T17:50:14.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.640+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~LZW6IEE3HT8M.1 size=472909 2024-03-04T17:50:14.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.640+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=LZW6IEE3HT8M tag=ANDZN5X0RHQ6 type=write size=2570061 2024-03-04T17:50:14.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.644+0000 7fe960f568c0 1 < completed write operation key=UEVMIPQ9FR81 tag=TN823R7QY7T4 type=write size=341090 2024-03-04T17:50:14.649 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.648+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=340B8LES4NJZ type=del size=3362666 2024-03-04T17:50:14.658 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.656+0000 7fe960f568c0 1 < suggested remove operation key=U2G6YEVD5506 tag=LH8T2PJF54EU type=write 2024-03-04T17:50:14.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.664+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~O18XM8MJG3D3.0 size=196870 2024-03-04T17:50:14.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.664+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=O18XM8MJG3D3 tag=69C5RN7UCI64 type=write size=1245446 2024-03-04T17:50:14.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.668+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~4BAEJGO5EWEU.0 size=1048576 2024-03-04T17:50:14.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.672+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~4BAEJGO5EWEU.1 size=1048576 2024-03-04T17:50:14.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.672+0000 7fe960f568c0 1 < canceled part key=_multipart_FLYDZHOIWO6L.2~4BAEJGO5EWEU.2 size=211062 2024-03-04T17:50:14.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.680+0000 7fe960f568c0 1 < canceled multipart upload key=FLYDZHOIWO6L upload=4BAEJGO5EWEU tag=EGU8DI15PKIL type=write size=3356790 2024-03-04T17:50:14.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.688+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~OSWBMFOQ9IC8.0 size=1048576 2024-03-04T17:50:14.691 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.692+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~OSWBMFOQ9IC8.1 size=28767 2024-03-04T17:50:14.691 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.692+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=OSWBMFOQ9IC8 tag=SQPP3WBZNZAK type=write size=2125919 2024-03-04T17:50:14.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.700+0000 7fe960f568c0 1 < suggested update operation key=2FUTMG0SSXBY tag=XKMLCQANY3JE type=del size=2796696 2024-03-04T17:50:14.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.704+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=31QHKXXKQ57K type=del size=564280 2024-03-04T17:50:14.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.708+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~OBI7V6LB2IMC.0 size=650500 2024-03-04T17:50:14.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.716+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=OBI7V6LB2IMC tag=YF6HUCQUC0AH type=write size=1699076 2024-03-04T17:50:14.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.720+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=RNGFEGXZOZBB type=write size=766676 2024-03-04T17:50:14.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.728+0000 7fe960f568c0 1 < suggested update operation key=JJ7SM1Y73A4C tag=KI86Q66HPDBW type=del size=2709416 2024-03-04T17:50:14.735 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.732+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=K9JNDFTLXAJH type=del size=3064504 2024-03-04T17:50:14.739 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.740+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=GA1XDC7GQRBZ type=del 2024-03-04T17:50:14.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.744+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=W0IXZN2WJWNZ type=del size=208877 2024-03-04T17:50:14.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.748+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~4DU7J0VS7PYX.0 size=1048576 2024-03-04T17:50:14.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.752+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~4DU7J0VS7PYX.1 size=1048576 2024-03-04T17:50:14.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.752+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~4DU7J0VS7PYX.2 size=361450 2024-03-04T17:50:14.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.760+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=4DU7J0VS7PYX tag=8RP63HX6XXPO type=write size=3507178 2024-03-04T17:50:14.767 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.764+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=GI6XGALTHCG0 type=write size=376494 2024-03-04T17:50:14.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.768+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=NUJ579B9482Q type=del 2024-03-04T17:50:14.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.772+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=AFBH2TEEGTVW type=del size=2601238 2024-03-04T17:50:14.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.776+0000 7fe960f568c0 1 < canceled operation key=3X4719GHK3BI tag=VBZO3H5GPINM type=write size=175041 2024-03-04T17:50:14.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.780+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=YGIZZDH1M9FT type=del size=3789135 2024-03-04T17:50:14.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.784+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=NPWTZR6VRKXW type=del 2024-03-04T17:50:14.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.788+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=BG8FYBQFLUVU type=del size=1688367 2024-03-04T17:50:14.797 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.796+0000 7fe960f568c0 1 < suggested remove operation key=UEVMIPQ9FR81 tag=28JV80VENGL3 type=del 2024-03-04T17:50:14.801 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.800+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=O2OVJ95WE4OT type=del size=3401994 2024-03-04T17:50:14.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.804+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=1N3QME40GO0B type=del 2024-03-04T17:50:14.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.812+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~ORI3TU1AN6U1.0 size=223975 2024-03-04T17:50:14.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.812+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=ORI3TU1AN6U1 tag=9SED8KYG3ZIB type=write size=1272551 2024-03-04T17:50:14.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.816+0000 7fe960f568c0 1 < suggested remove operation key=2MKQA5JYWU8L tag=6XIV2IB3323W type=del 2024-03-04T17:50:14.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.824+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~N6VYZPLJ8KVT.0 size=1048576 2024-03-04T17:50:14.828 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.828+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~N6VYZPLJ8KVT.1 size=1046871 2024-03-04T17:50:14.828 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.828+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=N6VYZPLJ8KVT tag=38KC7U63KE03 type=write size=3144023 2024-03-04T17:50:14.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.832+0000 7fe960f568c0 1 < completed write operation key=78N8HHC4O4CG tag=T933H4UX6QYK type=write size=929927 2024-03-04T17:50:14.835 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.836+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=9WU664XR0N2T type=del size=3194958 2024-03-04T17:50:14.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.840+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~LZW6IEE3HT8M.0 size=1048576 2024-03-04T17:50:14.842 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.840+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~LZW6IEE3HT8M.1 size=472909 2024-03-04T17:50:14.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.848+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=LZW6IEE3HT8M tag=ANDZN5X0RHQ6 type=write size=2570061 2024-03-04T17:50:14.855 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.852+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~CWVUV09MXRUD.0 size=1048576 2024-03-04T17:50:14.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.856+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~CWVUV09MXRUD.1 size=1048576 2024-03-04T17:50:14.861 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.860+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~CWVUV09MXRUD.2 size=828461 2024-03-04T17:50:14.861 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.860+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=CWVUV09MXRUD tag=55MCWR404A85 type=write size=3974189 2024-03-04T17:50:14.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.864+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=340B8LES4NJZ type=del 2024-03-04T17:50:14.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.872+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~40W799PZ0KOR.0 size=1048576 2024-03-04T17:50:14.875 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.872+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~40W799PZ0KOR.1 size=1048576 2024-03-04T17:50:14.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.876+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~40W799PZ0KOR.2 size=452911 2024-03-04T17:50:14.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.876+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=40W799PZ0KOR tag=5IMG9I3OVM4K type=write size=3598639 2024-03-04T17:50:14.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.880+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~O18XM8MJG3D3.0 size=196870 2024-03-04T17:50:14.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.888+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=O18XM8MJG3D3 tag=69C5RN7UCI64 type=write size=1245446 2024-03-04T17:50:14.892 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.892+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=RKXU6VMKMYJ2 type=del size=2621238 2024-03-04T17:50:14.893 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.892+0000 7fe960f568c0 1 < canceled part key=_multipart_2MKQA5JYWU8L.2~OSWBMFOQ9IC8.0 size=1048576 2024-03-04T17:50:14.893 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.892+0000 7fe960f568c0 1 < canceled part key=_multipart_2MKQA5JYWU8L.2~OSWBMFOQ9IC8.1 size=28767 2024-03-04T17:50:14.900 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.900+0000 7fe960f568c0 1 < canceled multipart upload key=2MKQA5JYWU8L upload=OSWBMFOQ9IC8 tag=SQPP3WBZNZAK type=write size=2125919 2024-03-04T17:50:14.904 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.904+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=NY6HLUWU18HN type=write size=160767 2024-03-04T17:50:14.908 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.908+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=31QHKXXKQ57K type=del 2024-03-04T17:50:14.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.916+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~ODXXBL4WJUAI.0 size=1048576 2024-03-04T17:50:14.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.916+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~ODXXBL4WJUAI.1 size=1048576 2024-03-04T17:50:14.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.920+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~ODXXBL4WJUAI.2 size=731038 2024-03-04T17:50:14.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.920+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=ODXXBL4WJUAI tag=W4DPJBISPM7N type=write size=3876766 2024-03-04T17:50:14.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.924+0000 7fe960f568c0 -1 < no bi entry to suggest for operation key=2MKQA5JYWU8L tag=RNGFEGXZOZBB type=write size=766676: (2) No such file or directory 2024-03-04T17:50:14.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.928+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=J1TPRMPZVPPO type=del size=3125500 2024-03-04T17:50:14.934 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.932+0000 7fe960f568c0 -1 < failed to complete operation key=2MKQA5JYWU8L tag=K9JNDFTLXAJH type=del size=3064504: (22) Invalid argument 2024-03-04T17:50:14.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.944+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~XJA7THF8HRI9.0 size=1048576 2024-03-04T17:50:14.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.944+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~XJA7THF8HRI9.1 size=1048576 2024-03-04T17:50:14.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.944+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~XJA7THF8HRI9.2 size=66927 2024-03-04T17:50:14.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.944+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=XJA7THF8HRI9 tag=OYUFKV3BC5OA type=write size=3212655 2024-03-04T17:50:14.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.952+0000 7fe960f568c0 1 < suggested remove operation key=WNXVI6K5MU3R tag=W0IXZN2WJWNZ type=del 2024-03-04T17:50:14.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.960+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~NEAJ3PGBMHAP.0 size=289148 2024-03-04T17:50:14.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.960+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=NEAJ3PGBMHAP tag=HX3CVHDQFAA1 type=write size=1337724 2024-03-04T17:50:14.964 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.964+0000 7fe960f568c0 -1 < failed to complete operation key=WNXVI6K5MU3R tag=GI6XGALTHCG0 type=write size=376494: (22) Invalid argument 2024-03-04T17:50:14.971 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.968+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~MTMFCNHAS01T.0 size=828212 2024-03-04T17:50:14.971 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.968+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=MTMFCNHAS01T tag=G5BIF202BOQD type=write size=1876788 2024-03-04T17:50:14.975 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.972+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=AFBH2TEEGTVW type=del 2024-03-04T17:50:14.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.980+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~AMN3DJ9YSDS2.0 size=1048576 2024-03-04T17:50:14.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.984+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~AMN3DJ9YSDS2.1 size=1014721 2024-03-04T17:50:14.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.984+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=AMN3DJ9YSDS2 tag=0ZPDD8I6NEI3 type=write size=3111873 2024-03-04T17:50:14.992 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.992+0000 7fe960f568c0 1 < suggested remove operation key=XSQ3GE2UHMZG tag=YGIZZDH1M9FT type=del 2024-03-04T17:50:14.995 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.996+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=Z674HJNB2IVQ type=write size=940822 2024-03-04T17:50:14.999 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:14.996+0000 7fe960f568c0 -1 < failed to complete operation key=XSQ3GE2UHMZG tag=BG8FYBQFLUVU type=del size=1688367: (22) Invalid argument 2024-03-04T17:50:15.003 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.000+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=26NH7XP79J0W type=write size=545484 2024-03-04T17:50:15.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.004+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=O2OVJ95WE4OT type=del 2024-03-04T17:50:15.013 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.012+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~7UTO6JQGN344.0 size=1048576 2024-03-04T17:50:15.015 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.016+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~7UTO6JQGN344.1 size=417136 2024-03-04T17:50:15.015 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.016+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=7UTO6JQGN344 tag=WPGF2Z1DZ6XA type=write size=2514288 2024-03-04T17:50:15.019 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.016+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~ORI3TU1AN6U1.0 size=223975 2024-03-04T17:50:15.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.024+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=ORI3TU1AN6U1 tag=9SED8KYG3ZIB type=write size=1272551 2024-03-04T17:50:15.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.028+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=172TARJCHPB4 type=del size=1495803 2024-03-04T17:50:15.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.032+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~N6VYZPLJ8KVT.0 size=1048576 2024-03-04T17:50:15.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.032+0000 7fe960f568c0 1 < canceled part key=_multipart_XSQ3GE2UHMZG.2~N6VYZPLJ8KVT.1 size=1046871 2024-03-04T17:50:15.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.040+0000 7fe960f568c0 1 < canceled multipart upload key=XSQ3GE2UHMZG upload=N6VYZPLJ8KVT tag=38KC7U63KE03 type=write size=3144023 2024-03-04T17:50:15.043 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.044+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=V40BSB6KFK6N type=del size=1136192 2024-03-04T17:50:15.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.048+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=9WU664XR0N2T type=del 2024-03-04T17:50:15.051 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.052+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=OYEI0IN0EKWM type=del size=1963712 2024-03-04T17:50:15.055 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.056+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~CWVUV09MXRUD.0 size=1048576 2024-03-04T17:50:15.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.056+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~CWVUV09MXRUD.1 size=1048576 2024-03-04T17:50:15.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.060+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~CWVUV09MXRUD.2 size=828461 2024-03-04T17:50:15.067 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.068+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=CWVUV09MXRUD tag=55MCWR404A85 type=write size=3974189 2024-03-04T17:50:15.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.068+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=XW5BA9ZZQ58V type=write size=796163 2024-03-04T17:50:15.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.072+0000 7fe960f568c0 1 < canceled part key=_multipart_2FUTMG0SSXBY.2~40W799PZ0KOR.0 size=1048576 2024-03-04T17:50:15.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.076+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~40W799PZ0KOR.1 size=1048576 2024-03-04T17:50:15.080 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.080+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~40W799PZ0KOR.2 size=452911 2024-03-04T17:50:15.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.084+0000 7fe960f568c0 1 < canceled multipart upload key=2FUTMG0SSXBY upload=40W799PZ0KOR tag=5IMG9I3OVM4K type=write size=3598639 2024-03-04T17:50:15.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.088+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=9DZEIZQJRW5O type=write size=65027 2024-03-04T17:50:15.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.092+0000 7fe960f568c0 1 < canceled operation key=KYSLL05AUHAC tag=RKXU6VMKMYJ2 type=del size=2621238 2024-03-04T17:50:15.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.100+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~XZR2QPAIG10E.0 size=1048576 2024-03-04T17:50:15.103 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.100+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~XZR2QPAIG10E.1 size=397011 2024-03-04T17:50:15.103 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.100+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=XZR2QPAIG10E tag=E7B2IPF5T3XH type=write size=2494163 2024-03-04T17:50:15.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.104+0000 7fe960f568c0 1 < completed write operation key=78N8HHC4O4CG tag=NY6HLUWU18HN type=write size=160767 2024-03-04T17:50:15.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.108+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=QQYZH539R7TP type=del size=2310922 2024-03-04T17:50:15.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.112+0000 7fe960f568c0 1 < canceled part key=_multipart_FLYDZHOIWO6L.2~ODXXBL4WJUAI.0 size=1048576 2024-03-04T17:50:15.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.112+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~ODXXBL4WJUAI.1 size=1048576 2024-03-04T17:50:15.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.116+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~ODXXBL4WJUAI.2 size=731038 2024-03-04T17:50:15.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.124+0000 7fe960f568c0 1 < canceled multipart upload key=FLYDZHOIWO6L upload=ODXXBL4WJUAI tag=W4DPJBISPM7N type=write size=3876766 2024-03-04T17:50:15.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.128+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~WNPJSR1TCZPT.0 size=1048576 2024-03-04T17:50:15.133 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.132+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~WNPJSR1TCZPT.1 size=1048576 2024-03-04T17:50:15.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.136+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~WNPJSR1TCZPT.2 size=673476 2024-03-04T17:50:15.136 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.136+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=WNPJSR1TCZPT tag=T9V1YG40H3A9 type=write size=3819204 2024-03-04T17:50:15.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.140+0000 7fe960f568c0 1 < canceled operation key=R6ZDVOUE9NXJ tag=J1TPRMPZVPPO type=del size=3125500 2024-03-04T17:50:15.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.144+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~FAHX00SJE1ZO.0 size=1048576 2024-03-04T17:50:15.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.148+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~FAHX00SJE1ZO.1 size=573437 2024-03-04T17:50:15.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.148+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=FAHX00SJE1ZO tag=1QV9IS9W0G71 type=write size=2670589 2024-03-04T17:50:15.153 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.152+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~XJA7THF8HRI9.0 size=1048576 2024-03-04T17:50:15.156 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.156+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~XJA7THF8HRI9.1 size=1048576 2024-03-04T17:50:15.159 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.156+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~XJA7THF8HRI9.2 size=66927 2024-03-04T17:50:15.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.164+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=XJA7THF8HRI9 tag=OYUFKV3BC5OA type=write size=3212655 2024-03-04T17:50:15.169 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.168+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=OR2XW4IJ3BIY type=del size=3856061 2024-03-04T17:50:15.173 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.172+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~NEAJ3PGBMHAP.0 size=289148 2024-03-04T17:50:15.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.176+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=NEAJ3PGBMHAP tag=HX3CVHDQFAA1 type=write size=1337724 2024-03-04T17:50:15.183 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.180+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=OEJQ48TA5FL7 type=del size=3485696 2024-03-04T17:50:15.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.184+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~MTMFCNHAS01T.0 size=828212 2024-03-04T17:50:15.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.192+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=MTMFCNHAS01T tag=G5BIF202BOQD type=write size=1876788 2024-03-04T17:50:15.196 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.196+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=WOLB05TLYLWZ type=del size=2593447 2024-03-04T17:50:15.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.200+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~AMN3DJ9YSDS2.0 size=1048576 2024-03-04T17:50:15.203 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.204+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~AMN3DJ9YSDS2.1 size=1014721 2024-03-04T17:50:15.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.208+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=AMN3DJ9YSDS2 tag=0ZPDD8I6NEI3 type=write size=3111873 2024-03-04T17:50:15.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.212+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=TIJBFKDWNDFR type=write size=931868 2024-03-04T17:50:15.217 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.216+0000 7fe960f568c0 1 < completed write operation key=MIZRUP4MEA0J tag=Z674HJNB2IVQ type=write size=940822 2024-03-04T17:50:15.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.224+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~HMZTT59615CI.0 size=214375 2024-03-04T17:50:15.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.224+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=HMZTT59615CI tag=CYMECK966WK6 type=write size=1262951 2024-03-04T17:50:15.228 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.228+0000 7fe960f568c0 1 < completed write operation key=FLYDZHOIWO6L tag=26NH7XP79J0W type=write size=545484 2024-03-04T17:50:15.231 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.232+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=IVRXQU8LP05K type=del size=412725 2024-03-04T17:50:15.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.236+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~7UTO6JQGN344.0 size=1048576 2024-03-04T17:50:15.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.236+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~7UTO6JQGN344.1 size=417136 2024-03-04T17:50:15.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.244+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=7UTO6JQGN344 tag=WPGF2Z1DZ6XA type=write size=2514288 2024-03-04T17:50:15.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.248+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=CPCC8ZKFS0YW type=del size=3674815 2024-03-04T17:50:15.252 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.252+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=172TARJCHPB4 type=del 2024-03-04T17:50:15.256 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.256+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=6030EAKX6S7R type=del size=3784003 2024-03-04T17:50:15.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.260+0000 7fe960f568c0 1 < suggested remove operation key=WNXVI6K5MU3R tag=V40BSB6KFK6N type=del 2024-03-04T17:50:15.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.264+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=BCKJWDMTAWIB type=del size=2415500 2024-03-04T17:50:15.276 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.276+0000 7fe960f568c0 1 < suggested remove operation key=UEVMIPQ9FR81 tag=OYEI0IN0EKWM type=del 2024-03-04T17:50:15.282 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.280+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~RIBJWYJQKBPF.0 size=476224 2024-03-04T17:50:15.282 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.280+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=RIBJWYJQKBPF tag=I3QTQA2OD03Y type=write size=1524800 2024-03-04T17:50:15.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.284+0000 7fe960f568c0 1 < canceled operation key=RII14ZGX2EL1 tag=XW5BA9ZZQ58V type=write size=796163 2024-03-04T17:50:15.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.288+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=Q0HVQEMHSTDV type=del size=2683502 2024-03-04T17:50:15.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.292+0000 7fe960f568c0 1 < canceled operation key=2FUTMG0SSXBY tag=9DZEIZQJRW5O type=write size=65027 2024-03-04T17:50:15.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.296+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=YTN4IKUCXA5P type=del size=354148 2024-03-04T17:50:15.301 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.300+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~XZR2QPAIG10E.0 size=1048576 2024-03-04T17:50:15.303 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.304+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~XZR2QPAIG10E.1 size=397011 2024-03-04T17:50:15.310 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.308+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=XZR2QPAIG10E tag=E7B2IPF5T3XH type=write size=2494163 2024-03-04T17:50:15.314 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.312+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=BAXXKU8EXQGL type=del size=1654269 2024-03-04T17:50:15.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.316+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=QQYZH539R7TP type=del 2024-03-04T17:50:15.326 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.324+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~5D3EV29SCJTU.0 size=1048576 2024-03-04T17:50:15.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.328+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~5D3EV29SCJTU.1 size=370634 2024-03-04T17:50:15.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.328+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=5D3EV29SCJTU tag=AGQNGGNV0FZ0 type=write size=2467786 2024-03-04T17:50:15.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.332+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~WNPJSR1TCZPT.0 size=1048576 2024-03-04T17:50:15.335 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.336+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~WNPJSR1TCZPT.1 size=1048576 2024-03-04T17:50:15.336 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.336+0000 7fe960f568c0 1 < canceled part key=_multipart_ASC735ZCBLCG.2~WNPJSR1TCZPT.2 size=673476 2024-03-04T17:50:15.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.340+0000 7fe960f568c0 1 < canceled multipart upload key=ASC735ZCBLCG upload=WNPJSR1TCZPT tag=T9V1YG40H3A9 type=write size=3819204 2024-03-04T17:50:15.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.348+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~85HF8J6WW6IC.0 size=1048576 2024-03-04T17:50:15.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.352+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~85HF8J6WW6IC.1 size=1048576 2024-03-04T17:50:15.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.352+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~85HF8J6WW6IC.2 size=704832 2024-03-04T17:50:15.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.352+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=85HF8J6WW6IC tag=5MNKZBPJKUQG type=write size=3850560 2024-03-04T17:50:15.359 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.360+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~FAHX00SJE1ZO.0 size=1048576 2024-03-04T17:50:15.362 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.360+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~FAHX00SJE1ZO.1 size=573437 2024-03-04T17:50:15.369 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.368+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=FAHX00SJE1ZO tag=1QV9IS9W0G71 type=write size=2670589 2024-03-04T17:50:15.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.376+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~JQV1SW37N5M9.0 size=1048576 2024-03-04T17:50:15.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.380+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~JQV1SW37N5M9.1 size=1019187 2024-03-04T17:50:15.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.380+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=JQV1SW37N5M9 tag=E9SAVGIV2NGY type=write size=3116339 2024-03-04T17:50:15.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.380+0000 7fe960f568c0 1 < canceled operation key=JJ7SM1Y73A4C tag=OR2XW4IJ3BIY type=del size=3856061 2024-03-04T17:50:15.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.384+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=BR0D4J5F43NT type=del size=940020 2024-03-04T17:50:15.391 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.388+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=OEJQ48TA5FL7 type=del 2024-03-04T17:50:15.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.392+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=K5QKC9FEM0KD type=del size=1380721 2024-03-04T17:50:15.399 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.396+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=WOLB05TLYLWZ type=del 2024-03-04T17:50:15.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.400+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=JO7DUFTWJFS1 type=del size=3007188 2024-03-04T17:50:15.406 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.404+0000 7fe960f568c0 1 < completed write operation key=P28N6U79ZUV2 tag=TIJBFKDWNDFR type=write size=931868 2024-03-04T17:50:15.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.412+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~DC361MQ3SAP2.0 size=1048576 2024-03-04T17:50:15.414 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.412+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~DC361MQ3SAP2.1 size=1048576 2024-03-04T17:50:15.417 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.416+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~DC361MQ3SAP2.2 size=740066 2024-03-04T17:50:15.417 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.416+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=DC361MQ3SAP2 tag=06MFPEMC7IR4 type=write size=3885794 2024-03-04T17:50:15.420 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.420+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~HMZTT59615CI.0 size=214375 2024-03-04T17:50:15.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.424+0000 7fe960f568c0 1 < completed multipart upload key=ASC735ZCBLCG upload=HMZTT59615CI tag=CYMECK966WK6 type=write size=1262951 2024-03-04T17:50:15.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.428+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=1GYATTOO6GTM type=del size=3565291 2024-03-04T17:50:15.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.432+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=IVRXQU8LP05K type=del 2024-03-04T17:50:15.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.440+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~1Y00VZWJ1U53.0 size=1048576 2024-03-04T17:50:15.442 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.440+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~1Y00VZWJ1U53.1 size=1048576 2024-03-04T17:50:15.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.444+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~1Y00VZWJ1U53.2 size=1537 2024-03-04T17:50:15.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.444+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=1Y00VZWJ1U53 tag=I3Q2J1YOPFJG type=write size=3147265 2024-03-04T17:50:15.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.448+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=CPCC8ZKFS0YW type=del 2024-03-04T17:50:15.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.456+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~P41H6J3W5KOB.0 size=1048576 2024-03-04T17:50:15.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.456+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~P41H6J3W5KOB.1 size=1048576 2024-03-04T17:50:15.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.460+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~P41H6J3W5KOB.2 size=724785 2024-03-04T17:50:15.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.460+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=P41H6J3W5KOB tag=R9JZ926USHJM type=write size=3870513 2024-03-04T17:50:15.468 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.468+0000 7fe960f568c0 1 < suggested remove operation key=8PZD01OOX3AF tag=6030EAKX6S7R type=del 2024-03-04T17:50:15.472 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.472+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=A4VFJTOZ7ODZ type=del size=695078 2024-03-04T17:50:15.476 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.476+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=BCKJWDMTAWIB type=del 2024-03-04T17:50:15.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.480+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=S0R8IBUIJKP7 type=del size=2090548 2024-03-04T17:50:15.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.480+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~RIBJWYJQKBPF.0 size=476224 2024-03-04T17:50:15.488 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.488+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=RIBJWYJQKBPF tag=I3QTQA2OD03Y type=write size=1524800 2024-03-04T17:50:15.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.492+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=LQN2JJHKS1HR type=del size=2122100 2024-03-04T17:50:15.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.496+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=Q0HVQEMHSTDV type=del 2024-03-04T17:50:15.499 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.496+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=0UTWS2HHRW4V type=del size=1842535 2024-03-04T17:50:15.503 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.500+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=YTN4IKUCXA5P type=del 2024-03-04T17:50:15.509 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.508+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~JC62L9Z0C7KJ.0 size=145813 2024-03-04T17:50:15.509 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.508+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=JC62L9Z0C7KJ tag=YOF9QEENP6TG type=write size=1194389 2024-03-04T17:50:15.513 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.512+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=BAXXKU8EXQGL type=del 2024-03-04T17:50:15.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.520+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~6TJF6ZTNNQWF.0 size=1048576 2024-03-04T17:50:15.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.520+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~6TJF6ZTNNQWF.1 size=1048576 2024-03-04T17:50:15.526 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.524+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~6TJF6ZTNNQWF.2 size=56373 2024-03-04T17:50:15.526 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.524+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=6TJF6ZTNNQWF tag=18YEBETFMX3K type=write size=3202101 2024-03-04T17:50:15.527 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.524+0000 7fe960f568c0 1 < canceled part key=_multipart_8PZD01OOX3AF.2~5D3EV29SCJTU.0 size=1048576 2024-03-04T17:50:15.530 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.528+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~5D3EV29SCJTU.1 size=370634 2024-03-04T17:50:15.536 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.536+0000 7fe960f568c0 1 < canceled multipart upload key=8PZD01OOX3AF upload=5D3EV29SCJTU tag=AGQNGGNV0FZ0 type=write size=2467786 2024-03-04T17:50:15.540 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.540+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=VGBLFZPV84X5 type=write size=333705 2024-03-04T17:50:15.544 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.544+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~85HF8J6WW6IC.0 size=1048576 2024-03-04T17:50:15.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.544+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~85HF8J6WW6IC.1 size=1048576 2024-03-04T17:50:15.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.544+0000 7fe960f568c0 1 < canceled part key=_multipart_QFF67JIW8Y8E.2~85HF8J6WW6IC.2 size=704832 2024-03-04T17:50:15.553 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.552+0000 7fe960f568c0 1 < canceled multipart upload key=QFF67JIW8Y8E upload=85HF8J6WW6IC tag=5MNKZBPJKUQG type=write size=3850560 2024-03-04T17:50:15.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.556+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=ZYVEEZMOCRV2 type=del size=1926813 2024-03-04T17:50:15.561 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.560+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~JQV1SW37N5M9.0 size=1048576 2024-03-04T17:50:15.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.564+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~JQV1SW37N5M9.1 size=1019187 2024-03-04T17:50:15.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.568+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=JQV1SW37N5M9 tag=E9SAVGIV2NGY type=write size=3116339 2024-03-04T17:50:15.576 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.576+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=D2X28S6CVJ0H type=del size=158209 2024-03-04T17:50:15.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.580+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=BR0D4J5F43NT type=del 2024-03-04T17:50:15.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.584+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=YLOPYVHWNB3P type=del size=898781 2024-03-04T17:50:15.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.588+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=K5QKC9FEM0KD type=del 2024-03-04T17:50:15.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.592+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~EZ1WDNB0Z9WI.0 size=116610 2024-03-04T17:50:15.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.592+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=EZ1WDNB0Z9WI tag=MLRZ0H8B258H type=write size=1165186 2024-03-04T17:50:15.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.596+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=JO7DUFTWJFS1 type=del 2024-03-04T17:50:15.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.600+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=XTT7CVR7GLUG type=del size=1054920 2024-03-04T17:50:15.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.604+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~DC361MQ3SAP2.0 size=1048576 2024-03-04T17:50:15.609 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.608+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~DC361MQ3SAP2.1 size=1048576 2024-03-04T17:50:15.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.612+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~DC361MQ3SAP2.2 size=740066 2024-03-04T17:50:15.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.620+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=DC361MQ3SAP2 tag=06MFPEMC7IR4 type=write size=3885794 2024-03-04T17:50:15.628 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.628+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~FYI2EC48JQ8I.0 size=1048576 2024-03-04T17:50:15.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.628+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~FYI2EC48JQ8I.1 size=1048576 2024-03-04T17:50:15.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.632+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~FYI2EC48JQ8I.2 size=923119 2024-03-04T17:50:15.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.632+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=FYI2EC48JQ8I tag=0XI23N74HR2T type=write size=4068847 2024-03-04T17:50:15.637 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.636+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=1GYATTOO6GTM type=del 2024-03-04T17:50:15.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.640+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=P8AYBOJ5PFK5 type=write size=602385 2024-03-04T17:50:15.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.644+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~1Y00VZWJ1U53.0 size=1048576 2024-03-04T17:50:15.648 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.648+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~1Y00VZWJ1U53.1 size=1048576 2024-03-04T17:50:15.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.648+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~1Y00VZWJ1U53.2 size=1537 2024-03-04T17:50:15.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.656+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=1Y00VZWJ1U53 tag=I3Q2J1YOPFJG type=write size=3147265 2024-03-04T17:50:15.663 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.664+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~A3PWSOVYV7IR.0 size=1048576 2024-03-04T17:50:15.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.664+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~A3PWSOVYV7IR.1 size=485674 2024-03-04T17:50:15.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.664+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=A3PWSOVYV7IR tag=1M1TFJB84A4N type=write size=2582826 2024-03-04T17:50:15.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.668+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~P41H6J3W5KOB.0 size=1048576 2024-03-04T17:50:15.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.672+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~P41H6J3W5KOB.1 size=1048576 2024-03-04T17:50:15.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.676+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~P41H6J3W5KOB.2 size=724785 2024-03-04T17:50:15.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.680+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=P41H6J3W5KOB tag=R9JZ926USHJM type=write size=3870513 2024-03-04T17:50:15.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.688+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~ZI25JXLDBEZR.0 size=730200 2024-03-04T17:50:15.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.688+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=ZI25JXLDBEZR tag=P1GBNW9CN5PJ type=write size=1778776 2024-03-04T17:50:15.693 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.692+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=A4VFJTOZ7ODZ type=del 2024-03-04T17:50:15.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.696+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=LB8IRXGHTGOE type=del size=2772287 2024-03-04T17:50:15.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.704+0000 7fe960f568c0 1 < suggested update operation key=P28N6U79ZUV2 tag=S0R8IBUIJKP7 type=del size=2090548 2024-03-04T17:50:15.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.708+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=VV781Y6KWVLW type=del size=4147875 2024-03-04T17:50:15.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.712+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=LQN2JJHKS1HR type=del 2024-03-04T17:50:15.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.716+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=LXY1EBBS1FJN type=del size=4113263 2024-03-04T17:50:15.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.720+0000 7fe960f568c0 1 < canceled operation key=WNXVI6K5MU3R tag=0UTWS2HHRW4V type=del size=1842535 2024-03-04T17:50:15.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.724+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=G74S8Y2SIHUJ type=del size=2236613 2024-03-04T17:50:15.728 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.728+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~JC62L9Z0C7KJ.0 size=145813 2024-03-04T17:50:15.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.732+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=JC62L9Z0C7KJ tag=YOF9QEENP6TG type=write size=1194389 2024-03-04T17:50:15.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.736+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=KPZ1LOGFVFC6 type=del size=2788696 2024-03-04T17:50:15.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.740+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~6TJF6ZTNNQWF.0 size=1048576 2024-03-04T17:50:15.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.744+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~6TJF6ZTNNQWF.1 size=1048576 2024-03-04T17:50:15.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.748+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~6TJF6ZTNNQWF.2 size=56373 2024-03-04T17:50:15.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.752+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=6TJF6ZTNNQWF tag=18YEBETFMX3K type=write size=3202101 2024-03-04T17:50:15.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.756+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=MIHIQ87CXI86 type=del size=2276386 2024-03-04T17:50:15.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.760+0000 7fe960f568c0 1 < completed write operation key=KYSLL05AUHAC tag=VGBLFZPV84X5 type=write size=333705 2024-03-04T17:50:15.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.764+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=BSL5GSGYWETR type=write size=938431 2024-03-04T17:50:15.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.768+0000 7fe960f568c0 1 < canceled operation key=BFF9OD00TU54 tag=ZYVEEZMOCRV2 type=del size=1926813 2024-03-04T17:50:15.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.772+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=4SNCT8RQM2BJ type=write size=758391 2024-03-04T17:50:15.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.780+0000 7fe960f568c0 1 < canceled operation key=BFF9OD00TU54 tag=D2X28S6CVJ0H type=del size=158209 2024-03-04T17:50:15.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.780+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=QY1LHQG1DOQI type=del size=2699720 2024-03-04T17:50:15.787 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.788+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=YLOPYVHWNB3P type=del 2024-03-04T17:50:15.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.792+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~KMXBMGAQGOIM.0 size=340902 2024-03-04T17:50:15.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.792+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=KMXBMGAQGOIM tag=0AOW5AXM2L13 type=write size=1389478 2024-03-04T17:50:15.798 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.796+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~EZ1WDNB0Z9WI.0 size=116610 2024-03-04T17:50:15.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.808+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=EZ1WDNB0Z9WI tag=MLRZ0H8B258H type=write size=1165186 2024-03-04T17:50:15.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.808+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~KEVNKPPFXRNG.0 size=1048576 2024-03-04T17:50:15.814 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.812+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~KEVNKPPFXRNG.1 size=980193 2024-03-04T17:50:15.814 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.812+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=KEVNKPPFXRNG tag=0ENORKDM6LN3 type=write size=3077345 2024-03-04T17:50:15.822 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.820+0000 7fe960f568c0 1 < suggested update operation key=QGLN27WH1N2O tag=XTT7CVR7GLUG type=del size=1054920 2024-03-04T17:50:15.831 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.828+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~K34LCKS7RZTZ.0 size=1048576 2024-03-04T17:50:15.834 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.832+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~K34LCKS7RZTZ.1 size=1048576 2024-03-04T17:50:15.836 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.836+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~K34LCKS7RZTZ.2 size=919300 2024-03-04T17:50:15.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.836+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=K34LCKS7RZTZ tag=FB1AIOGB8E72 type=write size=4065028 2024-03-04T17:50:15.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.840+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~FYI2EC48JQ8I.0 size=1048576 2024-03-04T17:50:15.843 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.844+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~FYI2EC48JQ8I.1 size=1048576 2024-03-04T17:50:15.846 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.844+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~FYI2EC48JQ8I.2 size=923119 2024-03-04T17:50:15.852 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.856+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=FYI2EC48JQ8I tag=0XI23N74HR2T type=write size=4068847 2024-03-04T17:50:15.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.856+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=QAHUKSRMJXDI type=del size=2021026 2024-03-04T17:50:15.860 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.860+0000 7fe960f568c0 1 < completed write operation key=7Z7BPZ8XGBHZ tag=P8AYBOJ5PFK5 type=write size=602385 2024-03-04T17:50:15.864 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.864+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=LCOTLC3AKQ94 type=write size=65920 2024-03-04T17:50:15.868 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.868+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~A3PWSOVYV7IR.0 size=1048576 2024-03-04T17:50:15.871 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.872+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~A3PWSOVYV7IR.1 size=485674 2024-03-04T17:50:15.877 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.876+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=A3PWSOVYV7IR tag=1M1TFJB84A4N type=write size=2582826 2024-03-04T17:50:15.885 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.884+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~PKXQD50ZBCMP.0 size=1048576 2024-03-04T17:50:15.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.888+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~PKXQD50ZBCMP.1 size=377424 2024-03-04T17:50:15.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.888+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=PKXQD50ZBCMP tag=O9Y0S6DH5A8U type=write size=2474576 2024-03-04T17:50:15.892 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.892+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~ZI25JXLDBEZR.0 size=730200 2024-03-04T17:50:15.900 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.900+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=ZI25JXLDBEZR tag=P1GBNW9CN5PJ type=write size=1778776 2024-03-04T17:50:15.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.908+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~FXNC99PJC97M.0 size=1048576 2024-03-04T17:50:15.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.908+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~FXNC99PJC97M.1 size=30936 2024-03-04T17:50:15.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.908+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=FXNC99PJC97M tag=28O0CVTTY67F type=write size=2128088 2024-03-04T17:50:15.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.912+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=LB8IRXGHTGOE type=del 2024-03-04T17:50:15.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.916+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=7A0Q4T13VOZJ type=del size=2813948 2024-03-04T17:50:15.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.920+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=VV781Y6KWVLW type=del 2024-03-04T17:50:15.926 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.924+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=0OERXCW9YU91 type=write size=124135 2024-03-04T17:50:15.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.928+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=LXY1EBBS1FJN type=del 2024-03-04T17:50:15.937 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.936+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~5C3WT6NKIG4G.0 size=1048576 2024-03-04T17:50:15.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.940+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~5C3WT6NKIG4G.1 size=233255 2024-03-04T17:50:15.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.940+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=5C3WT6NKIG4G tag=8BZ2JG0TVRXK type=write size=2330407 2024-03-04T17:50:15.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.944+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=G74S8Y2SIHUJ type=del 2024-03-04T17:50:15.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.948+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=JIX0AV46DMSQ type=del size=3293856 2024-03-04T17:50:15.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.952+0000 7fe960f568c0 -1 < failed to complete operation key=QGLN27WH1N2O tag=KPZ1LOGFVFC6 type=del size=2788696: (22) Invalid argument 2024-03-04T17:50:15.960 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.960+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~F9S5HTMMUDHH.0 size=1048576 2024-03-04T17:50:15.963 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.960+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~F9S5HTMMUDHH.1 size=1048576 2024-03-04T17:50:15.966 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.964+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~F9S5HTMMUDHH.2 size=348751 2024-03-04T17:50:15.966 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.964+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=F9S5HTMMUDHH tag=Y6D03827KVII type=write size=3494479 2024-03-04T17:50:15.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.968+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=MIHIQ87CXI86 type=del 2024-03-04T17:50:15.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.972+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=YKTXLZWBDT02 type=del size=2800115 2024-03-04T17:50:15.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.980+0000 7fe960f568c0 1 < completed write operation key=U2G6YEVD5506 tag=BSL5GSGYWETR type=write size=938431 2024-03-04T17:50:15.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.984+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=O02EID5VUNHS type=del size=4013918 2024-03-04T17:50:15.988 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.988+0000 7fe960f568c0 1 < completed write operation key=QFF67JIW8Y8E tag=4SNCT8RQM2BJ type=write size=758391 2024-03-04T17:50:15.995 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.992+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~LH4GI5542CDA.0 size=395758 2024-03-04T17:50:15.995 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:15.992+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=LH4GI5542CDA tag=WRAO6REJGDXM type=write size=1444334 2024-03-04T17:50:16.001 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.000+0000 7fe960f568c0 1 < suggested remove operation key=ASC735ZCBLCG tag=QY1LHQG1DOQI type=del 2024-03-04T17:50:16.004 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.004+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=9EHY2TKU239B type=del size=418927 2024-03-04T17:50:16.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.008+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~KMXBMGAQGOIM.0 size=340902 2024-03-04T17:50:16.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.012+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=KMXBMGAQGOIM tag=0AOW5AXM2L13 type=write size=1389478 2024-03-04T17:50:16.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.016+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=FO14JRH8GSM6 type=del size=3642365 2024-03-04T17:50:16.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.020+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~KEVNKPPFXRNG.0 size=1048576 2024-03-04T17:50:16.023 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.024+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~KEVNKPPFXRNG.1 size=980193 2024-03-04T17:50:16.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.028+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=KEVNKPPFXRNG tag=0ENORKDM6LN3 type=write size=3077345 2024-03-04T17:50:16.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.036+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~VOEPGYL2JEMP.0 size=1048576 2024-03-04T17:50:16.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.036+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~VOEPGYL2JEMP.1 size=277525 2024-03-04T17:50:16.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.036+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=VOEPGYL2JEMP tag=ZIY8GYV93I7P type=write size=2374677 2024-03-04T17:50:16.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.040+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~K34LCKS7RZTZ.0 size=1048576 2024-03-04T17:50:16.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.040+0000 7fe960f568c0 1 < canceled part key=_multipart_UKZ079SVZZKG.2~K34LCKS7RZTZ.1 size=1048576 2024-03-04T17:50:16.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.044+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~K34LCKS7RZTZ.2 size=919300 2024-03-04T17:50:16.051 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.048+0000 7fe960f568c0 1 < canceled multipart upload key=UKZ079SVZZKG upload=K34LCKS7RZTZ tag=FB1AIOGB8E72 type=write size=4065028 2024-03-04T17:50:16.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.056+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~RUXOBFW35LCR.0 size=1048576 2024-03-04T17:50:16.060 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.060+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~RUXOBFW35LCR.1 size=999371 2024-03-04T17:50:16.060 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.060+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=RUXOBFW35LCR tag=2C7AE61SGZV2 type=write size=3096523 2024-03-04T17:50:16.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.064+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=QAHUKSRMJXDI type=del 2024-03-04T17:50:16.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.068+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=2AAESR80HVOF type=del size=2257776 2024-03-04T17:50:16.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.072+0000 7fe960f568c0 1 < completed write operation key=2KZ4IFW54C5B tag=LCOTLC3AKQ94 type=write size=65920 2024-03-04T17:50:16.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.076+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=3CVY2QSCKRAA type=write size=455695 2024-03-04T17:50:16.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.076+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~PKXQD50ZBCMP.0 size=1048576 2024-03-04T17:50:16.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.076+0000 7fe960f568c0 1 < canceled part key=_multipart_3X4719GHK3BI.2~PKXQD50ZBCMP.1 size=377424 2024-03-04T17:50:16.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.084+0000 7fe960f568c0 1 < canceled multipart upload key=3X4719GHK3BI upload=PKXQD50ZBCMP tag=O9Y0S6DH5A8U type=write size=2474576 2024-03-04T17:50:16.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.088+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=ONFB2GK2Z0SL type=del size=442085 2024-03-04T17:50:16.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.088+0000 7fe960f568c0 1 < canceled part key=_multipart_7Z7BPZ8XGBHZ.2~FXNC99PJC97M.0 size=1048576 2024-03-04T17:50:16.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.092+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~FXNC99PJC97M.1 size=30936 2024-03-04T17:50:16.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.100+0000 7fe960f568c0 1 < canceled multipart upload key=7Z7BPZ8XGBHZ upload=FXNC99PJC97M tag=28O0CVTTY67F type=write size=2128088 2024-03-04T17:50:16.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.104+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~BW2UTD28TAAC.0 size=178020 2024-03-04T17:50:16.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.104+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=BW2UTD28TAAC tag=8JFH8DAA4QN7 type=write size=1226596 2024-03-04T17:50:16.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.108+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=7A0Q4T13VOZJ type=del 2024-03-04T17:50:16.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.116+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=RUZDX4W1DO8W type=del size=1803981 2024-03-04T17:50:16.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.120+0000 7fe960f568c0 1 < completed write operation key=3YTAYUQLLUKY tag=0OERXCW9YU91 type=write size=124135 2024-03-04T17:50:16.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.124+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=83H3FYJ439IE type=del size=3399662 2024-03-04T17:50:16.128 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.128+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~5C3WT6NKIG4G.0 size=1048576 2024-03-04T17:50:16.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.132+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~5C3WT6NKIG4G.1 size=233255 2024-03-04T17:50:16.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.136+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=5C3WT6NKIG4G tag=8BZ2JG0TVRXK type=write size=2330407 2024-03-04T17:50:16.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.144+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~MO5NAZ33JOJZ.0 size=1048576 2024-03-04T17:50:16.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.144+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~MO5NAZ33JOJZ.1 size=1048576 2024-03-04T17:50:16.150 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.148+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~MO5NAZ33JOJZ.2 size=549122 2024-03-04T17:50:16.150 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.148+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=MO5NAZ33JOJZ tag=0X93TRIJNNG0 type=write size=3694850 2024-03-04T17:50:16.153 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.152+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=JIX0AV46DMSQ type=del 2024-03-04T17:50:16.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.156+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=OD7QVWI4KLRU type=del size=2412920 2024-03-04T17:50:16.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.160+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~F9S5HTMMUDHH.0 size=1048576 2024-03-04T17:50:16.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.164+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~F9S5HTMMUDHH.1 size=1048576 2024-03-04T17:50:16.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.164+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~F9S5HTMMUDHH.2 size=348751 2024-03-04T17:50:16.174 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.172+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=F9S5HTMMUDHH tag=Y6D03827KVII type=write size=3494479 2024-03-04T17:50:16.181 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.180+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~GLG2POIJ7Y4H.0 size=1048576 2024-03-04T17:50:16.184 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.184+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~GLG2POIJ7Y4H.1 size=595928 2024-03-04T17:50:16.184 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.184+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=GLG2POIJ7Y4H tag=UU14TRLSJ5DL type=write size=2693080 2024-03-04T17:50:16.188 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.188+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=YKTXLZWBDT02 type=del 2024-03-04T17:50:16.191 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.192+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=6XKRWEUT1MZQ type=del size=1472201 2024-03-04T17:50:16.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.192+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=O02EID5VUNHS type=del 2024-03-04T17:50:16.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.196+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=IUD6EYELT998 type=del size=444031 2024-03-04T17:50:16.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.200+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~LH4GI5542CDA.0 size=395758 2024-03-04T17:50:16.208 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.208+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=LH4GI5542CDA tag=WRAO6REJGDXM type=write size=1444334 2024-03-04T17:50:16.215 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.216+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~5I0GXUVV0YAR.0 size=25970 2024-03-04T17:50:16.215 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.216+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=5I0GXUVV0YAR tag=05Q4EY4OFPIS type=write size=1074546 2024-03-04T17:50:16.219 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.216+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=9EHY2TKU239B type=del 2024-03-04T17:50:16.226 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.224+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~9WV44AEBLM3R.0 size=1048576 2024-03-04T17:50:16.229 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.228+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~9WV44AEBLM3R.1 size=1048576 2024-03-04T17:50:16.231 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.232+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~9WV44AEBLM3R.2 size=870764 2024-03-04T17:50:16.231 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.232+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=9WV44AEBLM3R tag=3LIL92MBYRUJ type=write size=4016492 2024-03-04T17:50:16.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.232+0000 7fe960f568c0 1 < canceled operation key=KYSLL05AUHAC tag=FO14JRH8GSM6 type=del size=3642365 2024-03-04T17:50:16.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.236+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=3AH1Z1ZYKMV0 type=write size=151879 2024-03-04T17:50:16.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.240+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~VOEPGYL2JEMP.0 size=1048576 2024-03-04T17:50:16.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.244+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~VOEPGYL2JEMP.1 size=277525 2024-03-04T17:50:16.253 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.252+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=VOEPGYL2JEMP tag=ZIY8GYV93I7P type=write size=2374677 2024-03-04T17:50:16.257 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.256+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=VTKJDNJMW79S type=write size=1027976 2024-03-04T17:50:16.261 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.260+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~RUXOBFW35LCR.0 size=1048576 2024-03-04T17:50:16.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.264+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~RUXOBFW35LCR.1 size=999371 2024-03-04T17:50:16.270 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.268+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=RUXOBFW35LCR tag=2C7AE61SGZV2 type=write size=3096523 2024-03-04T17:50:16.277 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.276+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~E806NN9NHTHF.0 size=1048576 2024-03-04T17:50:16.279 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.280+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~E806NN9NHTHF.1 size=1048576 2024-03-04T17:50:16.282 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.280+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~E806NN9NHTHF.2 size=116801 2024-03-04T17:50:16.283 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.280+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=E806NN9NHTHF tag=KS53DK2G0K8W type=write size=3262529 2024-03-04T17:50:16.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.284+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=2AAESR80HVOF type=del 2024-03-04T17:50:16.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.292+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~PA4A51570OS2.0 size=1048576 2024-03-04T17:50:16.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.296+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~PA4A51570OS2.1 size=464390 2024-03-04T17:50:16.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.296+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=PA4A51570OS2 tag=7W8VWZH068SD type=write size=2561542 2024-03-04T17:50:16.300 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.300+0000 7fe960f568c0 1 < completed write operation key=2KZ4IFW54C5B tag=3CVY2QSCKRAA type=write size=455695 2024-03-04T17:50:16.306 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.304+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~QBN2YW6A3RTU.0 size=1048576 2024-03-04T17:50:16.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.308+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~QBN2YW6A3RTU.1 size=1048576 2024-03-04T17:50:16.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.312+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~QBN2YW6A3RTU.2 size=740268 2024-03-04T17:50:16.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.312+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=QBN2YW6A3RTU tag=AJKXPEKK0OG2 type=write size=3885996 2024-03-04T17:50:16.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.320+0000 7fe960f568c0 1 < suggested update operation key=3YTAYUQLLUKY tag=ONFB2GK2Z0SL type=del size=442085 2024-03-04T17:50:16.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.328+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=B7TNY9JF8AJ0 type=write size=66578 2024-03-04T17:50:16.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.328+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~BW2UTD28TAAC.0 size=178020 2024-03-04T17:50:16.335 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.332+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=BW2UTD28TAAC tag=8JFH8DAA4QN7 type=write size=1226596 2024-03-04T17:50:16.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.336+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=IUE0TNE0L62R type=del size=3580762 2024-03-04T17:50:16.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.340+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=RUZDX4W1DO8W type=del 2024-03-04T17:50:16.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.344+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=GTXOZ2XZM95L type=del size=596755 2024-03-04T17:50:16.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.348+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=83H3FYJ439IE type=del 2024-03-04T17:50:16.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.352+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=EOU71N0SOM3H type=write size=759216 2024-03-04T17:50:16.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.356+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~MO5NAZ33JOJZ.0 size=1048576 2024-03-04T17:50:16.359 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.356+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~MO5NAZ33JOJZ.1 size=1048576 2024-03-04T17:50:16.362 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.360+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~MO5NAZ33JOJZ.2 size=549122 2024-03-04T17:50:16.367 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.368+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=MO5NAZ33JOJZ tag=0X93TRIJNNG0 type=write size=3694850 2024-03-04T17:50:16.371 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.368+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=B3M6FAVEUP7F type=del size=485375 2024-03-04T17:50:16.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.376+0000 7fe960f568c0 1 < suggested update operation key=JT02WL9JYTNM tag=OD7QVWI4KLRU type=del size=2412920 2024-03-04T17:50:16.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.380+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=O799ZW4Z9S52 type=del size=651925 2024-03-04T17:50:16.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.384+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~GLG2POIJ7Y4H.0 size=1048576 2024-03-04T17:50:16.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.388+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~GLG2POIJ7Y4H.1 size=595928 2024-03-04T17:50:16.394 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.392+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=GLG2POIJ7Y4H tag=UU14TRLSJ5DL type=write size=2693080 2024-03-04T17:50:16.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.396+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=0XNJUPHPT633 type=del size=270917 2024-03-04T17:50:16.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.400+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=6XKRWEUT1MZQ type=del 2024-03-04T17:50:16.406 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.404+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=R1VW4XMGGYI7 type=write size=830055 2024-03-04T17:50:16.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.408+0000 7fe960f568c0 -1 < failed to complete operation key=JT02WL9JYTNM tag=IUD6EYELT998 type=del size=444031: (22) Invalid argument 2024-03-04T17:50:16.413 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.412+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=EK6A8NMLN8SP type=del size=3707220 2024-03-04T17:50:16.417 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.416+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~5I0GXUVV0YAR.0 size=25970 2024-03-04T17:50:16.423 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.420+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=5I0GXUVV0YAR tag=05Q4EY4OFPIS type=write size=1074546 2024-03-04T17:50:16.427 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.424+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=T319SMNV9FO3 type=del size=3845265 2024-03-04T17:50:16.430 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.428+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~9WV44AEBLM3R.0 size=1048576 2024-03-04T17:50:16.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.432+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~9WV44AEBLM3R.1 size=1048576 2024-03-04T17:50:16.436 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.436+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~9WV44AEBLM3R.2 size=870764 2024-03-04T17:50:16.441 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.440+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=9WV44AEBLM3R tag=3LIL92MBYRUJ type=write size=4016492 2024-03-04T17:50:16.444 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.444+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=5PJJG2AHBSDX type=del size=666406 2024-03-04T17:50:16.451 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.452+0000 7fe960f568c0 1 < suggested update operation key=U2G6YEVD5506 tag=3AH1Z1ZYKMV0 type=write size=151879 2024-03-04T17:50:16.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.456+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~P9JW4B9WY8Z6.0 size=1048576 2024-03-04T17:50:16.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.464+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~P9JW4B9WY8Z6.1 size=804045 2024-03-04T17:50:16.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.464+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=P9JW4B9WY8Z6 tag=4SA9DN3UT10N type=write size=2901197 2024-03-04T17:50:16.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.464+0000 7fe960f568c0 1 < completed write operation key=UKZ079SVZZKG tag=VTKJDNJMW79S type=write size=1027976 2024-03-04T17:50:16.467 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.468+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=K9CJQT1KN149 type=del size=3790256 2024-03-04T17:50:16.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.472+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~E806NN9NHTHF.0 size=1048576 2024-03-04T17:50:16.474 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.472+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~E806NN9NHTHF.1 size=1048576 2024-03-04T17:50:16.478 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.476+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~E806NN9NHTHF.2 size=116801 2024-03-04T17:50:16.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.484+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=E806NN9NHTHF tag=KS53DK2G0K8W type=write size=3262529 2024-03-04T17:50:16.488 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.488+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=1AJK37NLLNPT type=write size=740906 2024-03-04T17:50:16.489 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.488+0000 7fe960f568c0 1 < canceled part key=_multipart_768X2XKWJW1S.2~PA4A51570OS2.0 size=1048576 2024-03-04T17:50:16.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.492+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~PA4A51570OS2.1 size=464390 2024-03-04T17:50:16.497 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.496+0000 7fe960f568c0 1 < canceled multipart upload key=768X2XKWJW1S upload=PA4A51570OS2 tag=7W8VWZH068SD type=write size=2561542 2024-03-04T17:50:16.504 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.504+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~PWDK596B6R2C.0 size=302623 2024-03-04T17:50:16.504 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.504+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=PWDK596B6R2C tag=0SGYJZQM8Q9P type=write size=1351199 2024-03-04T17:50:16.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.508+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~QBN2YW6A3RTU.0 size=1048576 2024-03-04T17:50:16.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.512+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~QBN2YW6A3RTU.1 size=1048576 2024-03-04T17:50:16.514 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.512+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~QBN2YW6A3RTU.2 size=740268 2024-03-04T17:50:16.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.520+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=QBN2YW6A3RTU tag=AJKXPEKK0OG2 type=write size=3885996 2024-03-04T17:50:16.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.524+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=ED02KM161QGO type=del size=3699919 2024-03-04T17:50:16.529 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.528+0000 7fe960f568c0 1 < completed write operation key=ASC735ZCBLCG tag=B7TNY9JF8AJ0 type=write size=66578 2024-03-04T17:50:16.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.532+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=J5GHEC9H2A5P type=write size=77687 2024-03-04T17:50:16.538 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.536+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=IUE0TNE0L62R type=del 2024-03-04T17:50:16.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.540+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=BGMKZ1SIRDSG type=del size=2102607 2024-03-04T17:50:16.546 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.544+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=GTXOZ2XZM95L type=del 2024-03-04T17:50:16.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.548+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=GSJUP4B7E1KI type=write size=637180 2024-03-04T17:50:16.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.556+0000 7fe960f568c0 1 < suggested remove operation key=GZ4RRIR5CC26 tag=EOU71N0SOM3H type=write 2024-03-04T17:50:16.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.560+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=RZ02S0IB1CGD type=del size=3581218 2024-03-04T17:50:16.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.568+0000 7fe960f568c0 1 < suggested remove operation key=FLYDZHOIWO6L tag=B3M6FAVEUP7F type=del 2024-03-04T17:50:16.574 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.572+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=1GU044TANVVY type=del size=4133094 2024-03-04T17:50:16.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.576+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=O799ZW4Z9S52 type=del 2024-03-04T17:50:16.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.580+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=XTSLLHHOJ1KX type=del size=465536 2024-03-04T17:50:16.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.584+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=0XNJUPHPT633 type=del 2024-03-04T17:50:16.590 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.588+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=VDY6ES1G200Q type=del size=2238211 2024-03-04T17:50:16.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.596+0000 7fe960f568c0 1 < suggested remove operation key=QGLN27WH1N2O tag=R1VW4XMGGYI7 type=write 2024-03-04T17:50:16.605 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.604+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~K3SFLVP9J10S.0 size=847355 2024-03-04T17:50:16.605 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.604+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=K3SFLVP9J10S tag=0TOE4ZNTSHPH type=write size=1895931 2024-03-04T17:50:16.609 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.608+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=EK6A8NMLN8SP type=del 2024-03-04T17:50:16.613 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.612+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=I2DBCSJBZDUM type=del size=633806 2024-03-04T17:50:16.617 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.616+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=T319SMNV9FO3 type=del 2024-03-04T17:50:16.624 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.624+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~TQIAOAZKUZIS.0 size=225781 2024-03-04T17:50:16.625 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.624+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=TQIAOAZKUZIS tag=LVF1DEN425J2 type=write size=1274357 2024-03-04T17:50:16.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.628+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=5PJJG2AHBSDX type=del 2024-03-04T17:50:16.633 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.632+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=5HKTUG7YEW4R type=del size=1300363 2024-03-04T17:50:16.638 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.636+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~P9JW4B9WY8Z6.0 size=1048576 2024-03-04T17:50:16.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.640+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~P9JW4B9WY8Z6.1 size=804045 2024-03-04T17:50:16.648 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.648+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=P9JW4B9WY8Z6 tag=4SA9DN3UT10N type=write size=2901197 2024-03-04T17:50:16.655 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.652+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~U2UV1RJXICXG.0 size=356931 2024-03-04T17:50:16.655 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.652+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=U2UV1RJXICXG tag=WC43XCZEQ4AD type=write size=1405507 2024-03-04T17:50:16.658 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.656+0000 7fe960f568c0 -1 < failed to complete operation key=GZ4RRIR5CC26 tag=K9CJQT1KN149 type=del size=3790256: (22) Invalid argument 2024-03-04T17:50:16.662 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.660+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=HS077V1S26IE type=write size=345283 2024-03-04T17:50:16.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.664+0000 7fe960f568c0 1 < completed write operation key=JSO6WIYMPCF6 tag=1AJK37NLLNPT type=write size=740906 2024-03-04T17:50:16.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.668+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=8S9F3FX98224 type=write size=429363 2024-03-04T17:50:16.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.672+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~PWDK596B6R2C.0 size=302623 2024-03-04T17:50:16.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.680+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=PWDK596B6R2C tag=0SGYJZQM8Q9P type=write size=1351199 2024-03-04T17:50:16.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.688+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=A0L4ZNNTGV2J type=write size=840208 2024-03-04T17:50:16.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.696+0000 7fe960f568c0 1 < suggested remove operation key=3YTAYUQLLUKY tag=ED02KM161QGO type=del 2024-03-04T17:50:16.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.700+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=76FY32293CYQ type=del size=219414 2024-03-04T17:50:16.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.704+0000 7fe960f568c0 1 < completed write operation key=8PZD01OOX3AF tag=J5GHEC9H2A5P type=write size=77687 2024-03-04T17:50:16.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.708+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=HYTBGERYSZ62 type=write size=704657 2024-03-04T17:50:16.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.708+0000 7fe960f568c0 -1 < no bi entry to suggest for operation key=GZ4RRIR5CC26 tag=BGMKZ1SIRDSG type=del size=2102607: (2) No such file or directory 2024-03-04T17:50:16.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.712+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=A7V9F935CIQ1 type=del size=1600642 2024-03-04T17:50:16.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.720+0000 7fe960f568c0 1 < completed write operation key=ASC735ZCBLCG tag=GSJUP4B7E1KI type=write size=637180 2024-03-04T17:50:16.728 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.728+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~EHIOV1DO48NL.0 size=115040 2024-03-04T17:50:16.728 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.728+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=EHIOV1DO48NL tag=Q3RLRPQLF1HB type=write size=1163616 2024-03-04T17:50:16.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.732+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=RZ02S0IB1CGD type=del 2024-03-04T17:50:16.736 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.736+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=96ZL0AM5CP3O type=del size=3008851 2024-03-04T17:50:16.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.744+0000 7fe960f568c0 1 < suggested remove operation key=FLYDZHOIWO6L tag=1GU044TANVVY type=del 2024-03-04T17:50:16.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.748+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=AXBO996BL3X2 type=del size=3198639 2024-03-04T17:50:16.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.760+0000 7fe960f568c0 1 < suggested remove operation key=BFF9OD00TU54 tag=XTSLLHHOJ1KX type=del 2024-03-04T17:50:16.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.760+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~JMW7BMFJAVHW.0 size=330885 2024-03-04T17:50:16.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.764+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=JMW7BMFJAVHW tag=6G4HZ5IGE74K type=write size=1379461 2024-03-04T17:50:16.767 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.768+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=VDY6ES1G200Q type=del 2024-03-04T17:50:16.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.772+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~20RQB7CDI9ZA.0 size=1048576 2024-03-04T17:50:16.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.776+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~20RQB7CDI9ZA.1 size=1048576 2024-03-04T17:50:16.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.780+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~20RQB7CDI9ZA.2 size=997640 2024-03-04T17:50:16.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.780+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=20RQB7CDI9ZA tag=IS0AFJND5I58 type=write size=4143368 2024-03-04T17:50:16.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.784+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~K3SFLVP9J10S.0 size=847355 2024-03-04T17:50:16.791 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.788+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=K3SFLVP9J10S tag=0TOE4ZNTSHPH type=write size=1895931 2024-03-04T17:50:16.797 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.796+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~8DUOB2682J8U.0 size=1048576 2024-03-04T17:50:16.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.800+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~8DUOB2682J8U.1 size=1048576 2024-03-04T17:50:16.803 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.804+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~8DUOB2682J8U.2 size=504020 2024-03-04T17:50:16.803 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.804+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=8DUOB2682J8U tag=A3EV1THY4M2L type=write size=3649748 2024-03-04T17:50:16.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.804+0000 7fe960f568c0 -1 < failed to complete operation key=3YTAYUQLLUKY tag=I2DBCSJBZDUM type=del size=633806: (22) Invalid argument 2024-03-04T17:50:16.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.808+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=RQIGF1S5ZUQZ type=write size=309124 2024-03-04T17:50:16.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.812+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~TQIAOAZKUZIS.0 size=225781 2024-03-04T17:50:16.820 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.820+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=TQIAOAZKUZIS tag=LVF1DEN425J2 type=write size=1274357 2024-03-04T17:50:16.827 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.828+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~BG9PX4MPOD65.0 size=833378 2024-03-04T17:50:16.828 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.828+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=BG9PX4MPOD65 tag=C32BISS72HL3 type=write size=1881954 2024-03-04T17:50:16.831 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.832+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=5HKTUG7YEW4R type=del 2024-03-04T17:50:16.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.836+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~JB0A0ZJWZUKU.0 size=1048576 2024-03-04T17:50:16.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.840+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~JB0A0ZJWZUKU.1 size=738665 2024-03-04T17:50:16.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.840+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=JB0A0ZJWZUKU tag=FAZ8JNNAOJVZ type=write size=2835817 2024-03-04T17:50:16.845 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.844+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~U2UV1RJXICXG.0 size=356931 2024-03-04T17:50:16.851 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.848+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=U2UV1RJXICXG tag=WC43XCZEQ4AD type=write size=1405507 2024-03-04T17:50:16.857 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.856+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~G1KQQDM6EJC6.0 size=1048576 2024-03-04T17:50:16.860 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.860+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~G1KQQDM6EJC6.1 size=323678 2024-03-04T17:50:16.860 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.860+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=G1KQQDM6EJC6 tag=0S0VWZ5IZQ6N type=write size=2420830 2024-03-04T17:50:16.864 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.864+0000 7fe960f568c0 1 < completed write operation key=JJ7SM1Y73A4C tag=HS077V1S26IE type=write size=345283 2024-03-04T17:50:16.868 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.868+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=8WG67I5RYRGY type=del size=1949314 2024-03-04T17:50:16.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.872+0000 7fe960f568c0 1 < completed write operation key=8PZD01OOX3AF tag=8S9F3FX98224 type=write size=429363 2024-03-04T17:50:16.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.876+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~A4PT1C4DAPU6.0 size=1048576 2024-03-04T17:50:16.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.880+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~A4PT1C4DAPU6.1 size=1048576 2024-03-04T17:50:16.884 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.884+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~A4PT1C4DAPU6.2 size=870038 2024-03-04T17:50:16.884 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.884+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=A4PT1C4DAPU6 tag=1TM4FZUVTNU3 type=write size=4015766 2024-03-04T17:50:16.892 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.892+0000 7fe960f568c0 1 < suggested update operation key=ASC735ZCBLCG tag=A0L4ZNNTGV2J type=write size=840208 2024-03-04T17:50:16.899 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.896+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~6XTX4ZOVEA23.0 size=1048576 2024-03-04T17:50:16.902 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.900+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~6XTX4ZOVEA23.1 size=598274 2024-03-04T17:50:16.902 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.900+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=6XTX4ZOVEA23 tag=2XMOIF9RBWLT type=write size=2695426 2024-03-04T17:50:16.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.904+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=76FY32293CYQ type=del 2024-03-04T17:50:16.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.912+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~RWN5LNO23MKH.0 size=1048576 2024-03-04T17:50:16.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.916+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~RWN5LNO23MKH.1 size=1048576 2024-03-04T17:50:16.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.916+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~RWN5LNO23MKH.2 size=12791 2024-03-04T17:50:16.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.916+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=RWN5LNO23MKH tag=0SONYRJ6NSO2 type=write size=3158519 2024-03-04T17:50:16.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.920+0000 7fe960f568c0 1 < completed write operation key=QGLN27WH1N2O tag=HYTBGERYSZ62 type=write size=704657 2024-03-04T17:50:16.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.928+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~8PQ83TB0OF72.0 size=1048576 2024-03-04T17:50:16.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.932+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~8PQ83TB0OF72.1 size=61390 2024-03-04T17:50:16.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.932+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=8PQ83TB0OF72 tag=25IF68OHBCK8 type=write size=2158542 2024-03-04T17:50:16.936 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.936+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=A7V9F935CIQ1 type=del 2024-03-04T17:50:16.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.940+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=YLQVQCI0QKMM type=del size=3696204 2024-03-04T17:50:16.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.944+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~EHIOV1DO48NL.0 size=115040 2024-03-04T17:50:16.950 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.948+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=EHIOV1DO48NL tag=Q3RLRPQLF1HB type=write size=1163616 2024-03-04T17:50:16.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.952+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=76OPCU8QVAC3 type=del size=244394 2024-03-04T17:50:16.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.956+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=96ZL0AM5CP3O type=del 2024-03-04T17:50:16.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.960+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=NRXLMDP1F8H9 type=del size=2226976 2024-03-04T17:50:16.969 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.968+0000 7fe960f568c0 1 < suggested remove operation key=R6ZDVOUE9NXJ tag=AXBO996BL3X2 type=del 2024-03-04T17:50:16.973 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.972+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=DIMVBTP69064 type=del size=2076285 2024-03-04T17:50:16.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.976+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~JMW7BMFJAVHW.0 size=330885 2024-03-04T17:50:16.983 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.984+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=JMW7BMFJAVHW tag=6G4HZ5IGE74K type=write size=1379461 2024-03-04T17:50:16.987 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.984+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=BLH86QEC4CTN type=del size=1339854 2024-03-04T17:50:16.991 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.988+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~20RQB7CDI9ZA.0 size=1048576 2024-03-04T17:50:16.994 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.992+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~20RQB7CDI9ZA.1 size=1048576 2024-03-04T17:50:16.997 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:16.996+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~20RQB7CDI9ZA.2 size=997640 2024-03-04T17:50:17.003 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.000+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=20RQB7CDI9ZA tag=IS0AFJND5I58 type=write size=4143368 2024-03-04T17:50:17.010 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.008+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~9FBFLAAOJE9E.0 size=63269 2024-03-04T17:50:17.010 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.008+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=9FBFLAAOJE9E tag=VY9RCG2FK408 type=write size=1111845 2024-03-04T17:50:17.013 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.012+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~8DUOB2682J8U.0 size=1048576 2024-03-04T17:50:17.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.012+0000 7fe960f568c0 1 < canceled part key=_multipart_3X4719GHK3BI.2~8DUOB2682J8U.1 size=1048576 2024-03-04T17:50:17.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.016+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~8DUOB2682J8U.2 size=504020 2024-03-04T17:50:17.022 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.020+0000 7fe960f568c0 1 < canceled multipart upload key=3X4719GHK3BI upload=8DUOB2682J8U tag=A3EV1THY4M2L type=write size=3649748 2024-03-04T17:50:17.027 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.024+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=10AR703NSHYY type=del size=3420869 2024-03-04T17:50:17.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.028+0000 7fe960f568c0 1 < completed write operation key=JT02WL9JYTNM tag=RQIGF1S5ZUQZ type=write size=309124 2024-03-04T17:50:17.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.040+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~20QMTS7BUYO2.0 size=1048576 2024-03-04T17:50:17.043 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.040+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~20QMTS7BUYO2.1 size=997194 2024-03-04T17:50:17.043 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.040+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=20QMTS7BUYO2 tag=6BW94DHGLT9X type=write size=3094346 2024-03-04T17:50:17.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.044+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~BG9PX4MPOD65.0 size=833378 2024-03-04T17:50:17.052 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.052+0000 7fe960f568c0 1 < completed multipart upload key=U2G6YEVD5506 upload=BG9PX4MPOD65 tag=C32BISS72HL3 type=write size=1881954 2024-03-04T17:50:17.059 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.056+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~Q0VA46J3MGKB.0 size=1048576 2024-03-04T17:50:17.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.060+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~Q0VA46J3MGKB.1 size=1048576 2024-03-04T17:50:17.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.064+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~Q0VA46J3MGKB.2 size=528624 2024-03-04T17:50:17.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.064+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=Q0VA46J3MGKB tag=Q3A16HSI3TKV type=write size=3674352 2024-03-04T17:50:17.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.068+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~JB0A0ZJWZUKU.0 size=1048576 2024-03-04T17:50:17.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.072+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~JB0A0ZJWZUKU.1 size=738665 2024-03-04T17:50:17.078 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.076+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=JB0A0ZJWZUKU tag=FAZ8JNNAOJVZ type=write size=2835817 2024-03-04T17:50:17.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.084+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~0IS48S10BY9N.0 size=1048576 2024-03-04T17:50:17.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.088+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~0IS48S10BY9N.1 size=217900 2024-03-04T17:50:17.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.088+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=0IS48S10BY9N tag=F7SC6WLQ61QF type=write size=2315052 2024-03-04T17:50:17.091 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.092+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~G1KQQDM6EJC6.0 size=1048576 2024-03-04T17:50:17.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.092+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~G1KQQDM6EJC6.1 size=323678 2024-03-04T17:50:17.101 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.100+0000 7fe960f568c0 1 < completed multipart upload key=ASC735ZCBLCG upload=G1KQQDM6EJC6 tag=0S0VWZ5IZQ6N type=write size=2420830 2024-03-04T17:50:17.104 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.104+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=QK80BKNWHIMM type=del size=3277680 2024-03-04T17:50:17.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.108+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=8WG67I5RYRGY type=del 2024-03-04T17:50:17.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.116+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~WT2L5S51DFYR.0 size=1048576 2024-03-04T17:50:17.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.116+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~WT2L5S51DFYR.1 size=706363 2024-03-04T17:50:17.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.116+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=WT2L5S51DFYR tag=WJZUGP42HTOC type=write size=2803515 2024-03-04T17:50:17.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.120+0000 7fe960f568c0 1 < canceled part key=_multipart_XSQ3GE2UHMZG.2~A4PT1C4DAPU6.0 size=1048576 2024-03-04T17:50:17.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.120+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~A4PT1C4DAPU6.1 size=1048576 2024-03-04T17:50:17.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.124+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~A4PT1C4DAPU6.2 size=870038 2024-03-04T17:50:17.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.132+0000 7fe960f568c0 1 < canceled multipart upload key=XSQ3GE2UHMZG upload=A4PT1C4DAPU6 tag=1TM4FZUVTNU3 type=write size=4015766 2024-03-04T17:50:17.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.136+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~241L2GSYJS97.0 size=1048576 2024-03-04T17:50:17.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.140+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~241L2GSYJS97.1 size=1048576 2024-03-04T17:50:17.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.144+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~241L2GSYJS97.2 size=631213 2024-03-04T17:50:17.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.144+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=241L2GSYJS97 tag=5ZSJDBHWQ494 type=write size=3776941 2024-03-04T17:50:17.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.144+0000 7fe960f568c0 1 < canceled part key=_multipart_QFF67JIW8Y8E.2~6XTX4ZOVEA23.0 size=1048576 2024-03-04T17:50:17.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.148+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~6XTX4ZOVEA23.1 size=598274 2024-03-04T17:50:17.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.152+0000 7fe960f568c0 1 < canceled multipart upload key=QFF67JIW8Y8E upload=6XTX4ZOVEA23 tag=2XMOIF9RBWLT type=write size=2695426 2024-03-04T17:50:17.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.156+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=PRJ9MP9MK7UF type=write size=630878 2024-03-04T17:50:17.159 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.156+0000 7fe960f568c0 1 < canceled part key=_multipart_2MKQA5JYWU8L.2~RWN5LNO23MKH.0 size=1048576 2024-03-04T17:50:17.162 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.160+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~RWN5LNO23MKH.1 size=1048576 2024-03-04T17:50:17.166 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.164+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~RWN5LNO23MKH.2 size=12791 2024-03-04T17:50:17.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.172+0000 7fe960f568c0 1 < canceled multipart upload key=2MKQA5JYWU8L upload=RWN5LNO23MKH tag=0SONYRJ6NSO2 type=write size=3158519 2024-03-04T17:50:17.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.180+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~9HQZLWR2GTI7.0 size=1048576 2024-03-04T17:50:17.183 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.180+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~9HQZLWR2GTI7.1 size=1048576 2024-03-04T17:50:17.185 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.184+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~9HQZLWR2GTI7.2 size=793640 2024-03-04T17:50:17.185 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.184+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=9HQZLWR2GTI7 tag=EBGPMVUGMGTC type=write size=3939368 2024-03-04T17:50:17.189 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.188+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~8PQ83TB0OF72.0 size=1048576 2024-03-04T17:50:17.192 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.192+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~8PQ83TB0OF72.1 size=61390 2024-03-04T17:50:17.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.196+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=8PQ83TB0OF72 tag=25IF68OHBCK8 type=write size=2158542 2024-03-04T17:50:17.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.200+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=GNAIO3XCN68F type=write size=901734 2024-03-04T17:50:17.208 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.208+0000 7fe960f568c0 1 < suggested remove operation key=XSQ3GE2UHMZG tag=YLQVQCI0QKMM type=del 2024-03-04T17:50:17.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.212+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=TA11LEU85S3F type=del size=4157682 2024-03-04T17:50:17.216 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.216+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=76OPCU8QVAC3 type=del 2024-03-04T17:50:17.219 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.220+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=4J1DOSE5HHHH type=del size=478927 2024-03-04T17:50:17.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.220+0000 7fe960f568c0 1 < completed delete operation key=ASC735ZCBLCG tag=NRXLMDP1F8H9 type=del 2024-03-04T17:50:17.229 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.228+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~GKUDGMXOOF92.0 size=980310 2024-03-04T17:50:17.229 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.228+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=GKUDGMXOOF92 tag=JO7OIUI223PQ type=write size=2028886 2024-03-04T17:50:17.233 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.232+0000 7fe960f568c0 1 < canceled operation key=JP3FRB6MD7B4 tag=DIMVBTP69064 type=del size=2076285 2024-03-04T17:50:17.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.236+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=V90P1W61N2Q7 type=del size=2720883 2024-03-04T17:50:17.241 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.240+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=BLH86QEC4CTN type=del 2024-03-04T17:50:17.244 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.244+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=XT512393OEIR type=write size=822233 2024-03-04T17:50:17.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.248+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~9FBFLAAOJE9E.0 size=63269 2024-03-04T17:50:17.254 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.252+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=9FBFLAAOJE9E tag=VY9RCG2FK408 type=write size=1111845 2024-03-04T17:50:17.260 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.260+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~VKW0R536AO61.0 size=1048576 2024-03-04T17:50:17.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.264+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~VKW0R536AO61.1 size=602175 2024-03-04T17:50:17.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.264+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=VKW0R536AO61 tag=HP3Q5EYS0A9C type=write size=2699327 2024-03-04T17:50:17.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.264+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=10AR703NSHYY type=del 2024-03-04T17:50:17.270 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.268+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=MEPD38ZOJC09 type=del size=3889383 2024-03-04T17:50:17.274 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.272+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~20QMTS7BUYO2.0 size=1048576 2024-03-04T17:50:17.276 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.276+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~20QMTS7BUYO2.1 size=997194 2024-03-04T17:50:17.282 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.280+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=20QMTS7BUYO2 tag=6BW94DHGLT9X type=write size=3094346 2024-03-04T17:50:17.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.288+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~B34XVB0O848G.0 size=1048576 2024-03-04T17:50:17.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.288+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~B34XVB0O848G.1 size=1048576 2024-03-04T17:50:17.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.292+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~B34XVB0O848G.2 size=4791 2024-03-04T17:50:17.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.292+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=B34XVB0O848G tag=0S894V4JZ2US type=write size=3150519 2024-03-04T17:50:17.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.296+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~Q0VA46J3MGKB.0 size=1048576 2024-03-04T17:50:17.300 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.300+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~Q0VA46J3MGKB.1 size=1048576 2024-03-04T17:50:17.303 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.300+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~Q0VA46J3MGKB.2 size=528624 2024-03-04T17:50:17.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.308+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=Q0VA46J3MGKB tag=Q3A16HSI3TKV type=write size=3674352 2024-03-04T17:50:17.315 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.316+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~0455R1FKORUD.0 size=929864 2024-03-04T17:50:17.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.316+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=0455R1FKORUD tag=EAW7YMXL0V6H type=write size=1978440 2024-03-04T17:50:17.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.316+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~0IS48S10BY9N.0 size=1048576 2024-03-04T17:50:17.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.320+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~0IS48S10BY9N.1 size=217900 2024-03-04T17:50:17.327 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.328+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=0IS48S10BY9N tag=F7SC6WLQ61QF type=write size=2315052 2024-03-04T17:50:17.334 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.332+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~2R3ZV5FAQNBP.0 size=1048576 2024-03-04T17:50:17.337 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.336+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~2R3ZV5FAQNBP.1 size=534546 2024-03-04T17:50:17.337 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.336+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=2R3ZV5FAQNBP tag=ZT9OTYJ9DEIR type=write size=2631698 2024-03-04T17:50:17.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.340+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=QK80BKNWHIMM type=del 2024-03-04T17:50:17.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.344+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=DYSG77VPQJWJ type=del size=2539374 2024-03-04T17:50:17.348 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.348+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~WT2L5S51DFYR.0 size=1048576 2024-03-04T17:50:17.351 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.352+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~WT2L5S51DFYR.1 size=706363 2024-03-04T17:50:17.358 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.356+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=WT2L5S51DFYR tag=WJZUGP42HTOC type=write size=2803515 2024-03-04T17:50:17.361 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.360+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=198UW3131IY2 type=del size=389143 2024-03-04T17:50:17.365 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.364+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~241L2GSYJS97.0 size=1048576 2024-03-04T17:50:17.371 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.372+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~241L2GSYJS97.1 size=1048576 2024-03-04T17:50:17.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.372+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~241L2GSYJS97.2 size=631213 2024-03-04T17:50:17.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.380+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=241L2GSYJS97 tag=5ZSJDBHWQ494 type=write size=3776941 2024-03-04T17:50:17.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.384+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=361KGP9C36E0 type=del size=2904275 2024-03-04T17:50:17.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.388+0000 7fe960f568c0 1 < completed write operation key=U2G6YEVD5506 tag=PRJ9MP9MK7UF type=write size=630878 2024-03-04T17:50:17.396 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.396+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~TUPPQTICIM6F.0 size=1048576 2024-03-04T17:50:17.399 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.396+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~TUPPQTICIM6F.1 size=976925 2024-03-04T17:50:17.399 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.396+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=TUPPQTICIM6F tag=YLY4113FW915 type=write size=3074077 2024-03-04T17:50:17.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.400+0000 7fe960f568c0 1 < canceled part key=_multipart_768X2XKWJW1S.2~9HQZLWR2GTI7.0 size=1048576 2024-03-04T17:50:17.403 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.400+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~9HQZLWR2GTI7.1 size=1048576 2024-03-04T17:50:17.403 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.400+0000 7fe960f568c0 1 < canceled part key=_multipart_768X2XKWJW1S.2~9HQZLWR2GTI7.2 size=793640 2024-03-04T17:50:17.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.408+0000 7fe960f568c0 1 < canceled multipart upload key=768X2XKWJW1S upload=9HQZLWR2GTI7 tag=EBGPMVUGMGTC type=write size=3939368 2024-03-04T17:50:17.413 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.412+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=USVNZPHWKYXM type=write size=702505 2024-03-04T17:50:17.417 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.416+0000 7fe960f568c0 1 < completed write operation key=QGLN27WH1N2O tag=GNAIO3XCN68F type=write size=901734 2024-03-04T17:50:17.421 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.420+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=U34OY13PG6VW type=del size=1945434 2024-03-04T17:50:17.425 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.424+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=TA11LEU85S3F type=del 2024-03-04T17:50:17.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.432+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~TR989MU80S4I.0 size=182391 2024-03-04T17:50:17.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.432+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=TR989MU80S4I tag=03EN7JE8VPOV type=write size=1230967 2024-03-04T17:50:17.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.432+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=4J1DOSE5HHHH type=del 2024-03-04T17:50:17.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.436+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=FIL7L8VHHZDH type=del size=2192547 2024-03-04T17:50:17.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.440+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~GKUDGMXOOF92.0 size=980310 2024-03-04T17:50:17.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.448+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=GKUDGMXOOF92 tag=JO7OIUI223PQ type=write size=2028886 2024-03-04T17:50:17.452 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.452+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=X77W7HWBTKP0 type=write size=135929 2024-03-04T17:50:17.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.456+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=V90P1W61N2Q7 type=del 2024-03-04T17:50:17.459 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.460+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=J8DNKXGW8FM2 type=write size=896615 2024-03-04T17:50:17.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.460+0000 7fe960f568c0 1 < completed write operation key=6X4ZV3BFZNDL tag=XT512393OEIR type=write size=822233 2024-03-04T17:50:17.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.468+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~DYAEL7BMV6DI.0 size=43634 2024-03-04T17:50:17.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.468+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=DYAEL7BMV6DI tag=EPARJWMTHMXF type=write size=1092210 2024-03-04T17:50:17.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.468+0000 7fe960f568c0 1 < canceled part key=_multipart_JP3FRB6MD7B4.2~VKW0R536AO61.0 size=1048576 2024-03-04T17:50:17.474 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.472+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~VKW0R536AO61.1 size=602175 2024-03-04T17:50:17.480 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.480+0000 7fe960f568c0 1 < canceled multipart upload key=JP3FRB6MD7B4 upload=VKW0R536AO61 tag=HP3Q5EYS0A9C type=write size=2699327 2024-03-04T17:50:17.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.484+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=0RS67ER8JWX1 type=del size=3842802 2024-03-04T17:50:17.488 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.488+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=MEPD38ZOJC09 type=del 2024-03-04T17:50:17.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.492+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=KCBJMMDEJ15M type=write size=550635 2024-03-04T17:50:17.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.496+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~B34XVB0O848G.0 size=1048576 2024-03-04T17:50:17.499 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.496+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~B34XVB0O848G.1 size=1048576 2024-03-04T17:50:17.502 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.500+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~B34XVB0O848G.2 size=4791 2024-03-04T17:50:17.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.508+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=B34XVB0O848G tag=0S894V4JZ2US type=write size=3150519 2024-03-04T17:50:17.512 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.512+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=VFJNAG2KCST3 type=del size=939921 2024-03-04T17:50:17.513 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.512+0000 7fe960f568c0 1 < canceled part key=_multipart_3YTAYUQLLUKY.2~0455R1FKORUD.0 size=929864 2024-03-04T17:50:17.518 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.516+0000 7fe960f568c0 1 < canceled multipart upload key=3YTAYUQLLUKY upload=0455R1FKORUD tag=EAW7YMXL0V6H type=write size=1978440 2024-03-04T17:50:17.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.520+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=G026J339HQ49 type=del size=600155 2024-03-04T17:50:17.526 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.524+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~2R3ZV5FAQNBP.0 size=1048576 2024-03-04T17:50:17.529 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.528+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~2R3ZV5FAQNBP.1 size=534546 2024-03-04T17:50:17.535 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.536+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=2R3ZV5FAQNBP tag=ZT9OTYJ9DEIR type=write size=2631698 2024-03-04T17:50:17.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.540+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~ZZPW8RH28C1E.0 size=125810 2024-03-04T17:50:17.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.540+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=ZZPW8RH28C1E tag=JE1HG61N9OJF type=write size=1174386 2024-03-04T17:50:17.546 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.544+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=DYSG77VPQJWJ type=del 2024-03-04T17:50:17.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.552+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~3U5FOHWRKYR9.0 size=1048576 2024-03-04T17:50:17.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.556+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~3U5FOHWRKYR9.1 size=1048576 2024-03-04T17:50:17.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.556+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~3U5FOHWRKYR9.2 size=654517 2024-03-04T17:50:17.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.556+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=3U5FOHWRKYR9 tag=8COPYY6DOXNA type=write size=3800245 2024-03-04T17:50:17.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.564+0000 7fe960f568c0 1 < suggested remove operation key=JP3FRB6MD7B4 tag=198UW3131IY2 type=del 2024-03-04T17:50:17.570 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.568+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=LOG6HVU7OBF3 type=del size=1324616 2024-03-04T17:50:17.573 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.572+0000 7fe960f568c0 -1 < failed to complete operation key=JP3FRB6MD7B4 tag=361KGP9C36E0 type=del size=2904275: (22) Invalid argument 2024-03-04T17:50:17.581 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.580+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~MX1OR2VU2PIR.0 size=1000069 2024-03-04T17:50:17.581 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.580+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=MX1OR2VU2PIR tag=3NZGNGAY57H0 type=write size=2048645 2024-03-04T17:50:17.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.580+0000 7fe960f568c0 1 < canceled part key=_multipart_UKZ079SVZZKG.2~TUPPQTICIM6F.0 size=1048576 2024-03-04T17:50:17.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.584+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~TUPPQTICIM6F.1 size=976925 2024-03-04T17:50:17.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.588+0000 7fe960f568c0 1 < canceled multipart upload key=UKZ079SVZZKG upload=TUPPQTICIM6F tag=YLY4113FW915 type=write size=3074077 2024-03-04T17:50:17.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.592+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=O1DYY9P5XNTF type=write size=898227 2024-03-04T17:50:17.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.600+0000 7fe960f568c0 1 < suggested remove operation key=2KZ4IFW54C5B tag=USVNZPHWKYXM type=write 2024-03-04T17:50:17.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.604+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=6GJ7F1DJ16YL type=del size=3115909 2024-03-04T17:50:17.610 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.608+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=U34OY13PG6VW type=del 2024-03-04T17:50:17.614 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.612+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=9JIF0DSXYBRA type=write size=887850 2024-03-04T17:50:17.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.616+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~TR989MU80S4I.0 size=182391 2024-03-04T17:50:17.625 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.624+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=TR989MU80S4I tag=03EN7JE8VPOV type=write size=1230967 2024-03-04T17:50:17.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.628+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=PBZDRLZ3B0GZ type=del size=2283494 2024-03-04T17:50:17.633 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.632+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=FIL7L8VHHZDH type=del 2024-03-04T17:50:17.637 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.636+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=D8MF7JU5M4H3 type=del size=1230632 2024-03-04T17:50:17.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.640+0000 7fe960f568c0 1 < completed write operation key=UKZ079SVZZKG tag=X77W7HWBTKP0 type=write size=135929 2024-03-04T17:50:17.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.644+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=OWNZGON2F005 type=del size=2765625 2024-03-04T17:50:17.649 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.648+0000 7fe960f568c0 1 < completed write operation key=768X2XKWJW1S tag=J8DNKXGW8FM2 type=write size=896615 2024-03-04T17:50:17.653 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.652+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=29XSG7W2L4LH type=del size=1826401 2024-03-04T17:50:17.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.656+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~DYAEL7BMV6DI.0 size=43634 2024-03-04T17:50:17.663 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.660+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=DYAEL7BMV6DI tag=EPARJWMTHMXF type=write size=1092210 2024-03-04T17:50:17.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.664+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=L00I5MX4SDSA type=del size=2627301 2024-03-04T17:50:17.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.672+0000 7fe960f568c0 1 < suggested remove operation key=QGLN27WH1N2O tag=0RS67ER8JWX1 type=del 2024-03-04T17:50:17.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.676+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=8ODQ1T5ABJFC type=write size=214623 2024-03-04T17:50:17.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.680+0000 7fe960f568c0 1 < completed write operation key=768X2XKWJW1S tag=KCBJMMDEJ15M type=write size=550635 2024-03-04T17:50:17.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.684+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=NTV58B623KEV type=del size=1257345 2024-03-04T17:50:17.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.688+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=VFJNAG2KCST3 type=del 2024-03-04T17:50:17.693 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.692+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=KEOXWHLAAKGH type=del size=2893020 2024-03-04T17:50:17.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.696+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=G026J339HQ49 type=del 2024-03-04T17:50:17.701 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.700+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=C3QS2NF10AEU type=del size=1276664 2024-03-04T17:50:17.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.704+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~ZZPW8RH28C1E.0 size=125810 2024-03-04T17:50:17.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.708+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=ZZPW8RH28C1E tag=JE1HG61N9OJF type=write size=1174386 2024-03-04T17:50:17.714 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.712+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=62K63RT8R5WF type=del size=3802426 2024-03-04T17:50:17.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.716+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~3U5FOHWRKYR9.0 size=1048576 2024-03-04T17:50:17.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.720+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~3U5FOHWRKYR9.1 size=1048576 2024-03-04T17:50:17.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.720+0000 7fe960f568c0 1 < canceled part key=_multipart_QFF67JIW8Y8E.2~3U5FOHWRKYR9.2 size=654517 2024-03-04T17:50:17.729 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.728+0000 7fe960f568c0 1 < canceled multipart upload key=QFF67JIW8Y8E upload=3U5FOHWRKYR9 tag=8COPYY6DOXNA type=write size=3800245 2024-03-04T17:50:17.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.732+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=TO6RAXDWXR3D type=write size=760451 2024-03-04T17:50:17.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.740+0000 7fe960f568c0 1 < suggested remove operation key=FSWBX523X9V5 tag=LOG6HVU7OBF3 type=del 2024-03-04T17:50:17.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.744+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=7EWRB9OG3QTA type=del size=2362319 2024-03-04T17:50:17.749 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.748+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~MX1OR2VU2PIR.0 size=1000069 2024-03-04T17:50:17.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.756+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=MX1OR2VU2PIR tag=3NZGNGAY57H0 type=write size=2048645 2024-03-04T17:50:17.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.760+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=TK3ZU6ZVKSSF type=del size=1876737 2024-03-04T17:50:17.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.764+0000 7fe960f568c0 1 < completed write operation key=2MKQA5JYWU8L tag=O1DYY9P5XNTF type=write size=898227 2024-03-04T17:50:17.767 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.768+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=DTCR49GWQKLD type=del size=70265 2024-03-04T17:50:17.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.768+0000 7fe960f568c0 1 < canceled operation key=JSO6WIYMPCF6 tag=6GJ7F1DJ16YL type=del size=3115909 2024-03-04T17:50:17.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.772+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=O7BV71N1RO7N type=del size=139799 2024-03-04T17:50:17.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.776+0000 7fe960f568c0 1 < canceled operation key=XSQ3GE2UHMZG tag=9JIF0DSXYBRA type=write size=887850 2024-03-04T17:50:17.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.784+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~8Y8HC2UYH5F6.0 size=1048576 2024-03-04T17:50:17.788 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.788+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~8Y8HC2UYH5F6.1 size=1048576 2024-03-04T17:50:17.791 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.788+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~8Y8HC2UYH5F6.2 size=1030270 2024-03-04T17:50:17.791 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.788+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=8Y8HC2UYH5F6 tag=NCWJJ4359I3E type=write size=4175998 2024-03-04T17:50:17.795 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.792+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=PBZDRLZ3B0GZ type=del 2024-03-04T17:50:17.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.800+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~955MDS8RWEF6.0 size=1048576 2024-03-04T17:50:17.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.804+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~955MDS8RWEF6.1 size=1048576 2024-03-04T17:50:17.808 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.808+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~955MDS8RWEF6.2 size=215494 2024-03-04T17:50:17.808 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.808+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=955MDS8RWEF6 tag=YW8ZKCTC51WY type=write size=3361222 2024-03-04T17:50:17.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.812+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=D8MF7JU5M4H3 type=del 2024-03-04T17:50:17.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.812+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=60BLTOCSLUQA type=del size=2600424 2024-03-04T17:50:17.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.816+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=OWNZGON2F005 type=del 2024-03-04T17:50:17.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.824+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~RJWLLMPIGS50.0 size=1048576 2024-03-04T17:50:17.828 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.828+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~RJWLLMPIGS50.1 size=1048576 2024-03-04T17:50:17.830 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.828+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~RJWLLMPIGS50.2 size=819012 2024-03-04T17:50:17.830 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.828+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=RJWLLMPIGS50 tag=OVT4U4VDZNG6 type=write size=3964740 2024-03-04T17:50:17.834 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.832+0000 7fe960f568c0 1 < canceled operation key=MIZRUP4MEA0J tag=29XSG7W2L4LH type=del size=1826401 2024-03-04T17:50:17.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.836+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=C4202RQQHMM0 type=del size=3455160 2024-03-04T17:50:17.845 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.844+0000 7fe960f568c0 1 < suggested update operation key=MIZRUP4MEA0J tag=L00I5MX4SDSA type=del size=2627301 2024-03-04T17:50:17.851 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.848+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~3P6BTH8AAXGY.0 size=834633 2024-03-04T17:50:17.851 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.848+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=3P6BTH8AAXGY tag=LD7ATFGOJTYY type=write size=1883209 2024-03-04T17:50:17.855 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.852+0000 7fe960f568c0 1 < completed write operation key=2KZ4IFW54C5B tag=8ODQ1T5ABJFC type=write size=214623 2024-03-04T17:50:17.859 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.856+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=T1O2V4CGS3RP type=del size=487925 2024-03-04T17:50:17.863 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.860+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=NTV58B623KEV type=del 2024-03-04T17:50:17.867 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.864+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=TOVD2O3TQQZJ type=del size=1955145 2024-03-04T17:50:17.875 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.872+0000 7fe960f568c0 1 < suggested update operation key=P28N6U79ZUV2 tag=KEOXWHLAAKGH type=del size=2893020 2024-03-04T17:50:17.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.876+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=8LJWZ78BNGEO type=del size=3199199 2024-03-04T17:50:17.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.880+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=C3QS2NF10AEU type=del 2024-03-04T17:50:17.887 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.884+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=F3K7QO7YR75Q type=del size=550488 2024-03-04T17:50:17.891 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.892+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=62K63RT8R5WF type=del 2024-03-04T17:50:17.895 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.892+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=MAN9T5ZUJJCP type=del size=762431 2024-03-04T17:50:17.902 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.900+0000 7fe960f568c0 1 < suggested update operation key=7Z7BPZ8XGBHZ tag=TO6RAXDWXR3D type=write size=760451 2024-03-04T17:50:17.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.908+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~2EC2C857EB1A.0 size=1048576 2024-03-04T17:50:17.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.912+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~2EC2C857EB1A.1 size=1048576 2024-03-04T17:50:17.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.912+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~2EC2C857EB1A.2 size=605085 2024-03-04T17:50:17.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.912+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=2EC2C857EB1A tag=O2QLKC2T0BUU type=write size=3750813 2024-03-04T17:50:17.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.916+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=7EWRB9OG3QTA type=del 2024-03-04T17:50:17.925 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.924+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~014V6DHGC6R2.0 size=207430 2024-03-04T17:50:17.925 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.924+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=014V6DHGC6R2 tag=RT0SU3MJB5P6 type=write size=1256006 2024-03-04T17:50:17.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.932+0000 7fe960f568c0 1 < suggested update operation key=7Z7BPZ8XGBHZ tag=TK3ZU6ZVKSSF type=del size=1876737 2024-03-04T17:50:17.936 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.936+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=DIE7BTSTSNGG type=del size=50307 2024-03-04T17:50:17.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.944+0000 7fe960f568c0 1 < suggested update operation key=JP3FRB6MD7B4 tag=DTCR49GWQKLD type=del size=70265 2024-03-04T17:50:17.947 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.944+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=Q4NDD4ZFW0VB type=del size=2336313 2024-03-04T17:50:17.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.948+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=O7BV71N1RO7N type=del 2024-03-04T17:50:17.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.952+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=K5ZHXZV3USRP type=del size=3324848 2024-03-04T17:50:17.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.956+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~8Y8HC2UYH5F6.0 size=1048576 2024-03-04T17:50:17.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.960+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~8Y8HC2UYH5F6.1 size=1048576 2024-03-04T17:50:17.964 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.964+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~8Y8HC2UYH5F6.2 size=1030270 2024-03-04T17:50:17.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.968+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=8Y8HC2UYH5F6 tag=NCWJJ4359I3E type=write size=4175998 2024-03-04T17:50:17.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.972+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=2ATMLAMWEFT2 type=del size=3864293 2024-03-04T17:50:17.978 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.976+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~955MDS8RWEF6.0 size=1048576 2024-03-04T17:50:17.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.980+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~955MDS8RWEF6.1 size=1048576 2024-03-04T17:50:17.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.980+0000 7fe960f568c0 1 < canceled part key=_multipart_6X4ZV3BFZNDL.2~955MDS8RWEF6.2 size=215494 2024-03-04T17:50:17.989 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.988+0000 7fe960f568c0 1 < canceled multipart upload key=6X4ZV3BFZNDL upload=955MDS8RWEF6 tag=YW8ZKCTC51WY type=write size=3361222 2024-03-04T17:50:17.992 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.992+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=I4DIDNW1VTUX type=del size=147420 2024-03-04T17:50:17.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:17.996+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=60BLTOCSLUQA type=del 2024-03-04T17:50:18.002 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.000+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~CEO48A45W17K.0 size=1048576 2024-03-04T17:50:18.005 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.004+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~CEO48A45W17K.1 size=1048576 2024-03-04T17:50:18.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.008+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~CEO48A45W17K.2 size=451487 2024-03-04T17:50:18.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.008+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=CEO48A45W17K tag=OZ8JVT062A0X type=write size=3597215 2024-03-04T17:50:18.012 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.012+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~RJWLLMPIGS50.0 size=1048576 2024-03-04T17:50:18.015 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.012+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~RJWLLMPIGS50.1 size=1048576 2024-03-04T17:50:18.015 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.012+0000 7fe960f568c0 1 < canceled part key=_multipart_U2G6YEVD5506.2~RJWLLMPIGS50.2 size=819012 2024-03-04T17:50:18.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.020+0000 7fe960f568c0 1 < canceled multipart upload key=U2G6YEVD5506 upload=RJWLLMPIGS50 tag=OVT4U4VDZNG6 type=write size=3964740 2024-03-04T17:50:18.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.024+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=QR5UEIFRMPTW type=del size=2714173 2024-03-04T17:50:18.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.028+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=C4202RQQHMM0 type=del 2024-03-04T17:50:18.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.036+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~Z212GLWQH7FX.0 size=1048576 2024-03-04T17:50:18.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.036+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~Z212GLWQH7FX.1 size=933782 2024-03-04T17:50:18.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.036+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=Z212GLWQH7FX tag=ZIV77TXY9F3U type=write size=3030934 2024-03-04T17:50:18.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.040+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~3P6BTH8AAXGY.0 size=834633 2024-03-04T17:50:18.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.048+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=3P6BTH8AAXGY tag=LD7ATFGOJTYY type=write size=1883209 2024-03-04T17:50:18.055 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.052+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~4AA6ZO7XMN8M.0 size=1048576 2024-03-04T17:50:18.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.056+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~4AA6ZO7XMN8M.1 size=125522 2024-03-04T17:50:18.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.056+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=4AA6ZO7XMN8M tag=V493JERBPD73 type=write size=2222674 2024-03-04T17:50:18.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.060+0000 7fe960f568c0 1 < canceled operation key=3X4719GHK3BI tag=T1O2V4CGS3RP type=del size=487925 2024-03-04T17:50:18.066 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.064+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=MVSV4JQ26OAM type=del size=4043223 2024-03-04T17:50:18.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.068+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=TOVD2O3TQQZJ type=del 2024-03-04T17:50:18.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.072+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=UJLVGNPTD0LI type=del size=1519430 2024-03-04T17:50:18.078 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.076+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=8LJWZ78BNGEO type=del 2024-03-04T17:50:18.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.084+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=Q5RQMTUSBNKW type=del size=214736 2024-03-04T17:50:18.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.088+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=F3K7QO7YR75Q type=del 2024-03-04T17:50:18.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.096+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~KDKGS5AR36KJ.0 size=929782 2024-03-04T17:50:18.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.096+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=KDKGS5AR36KJ tag=PLWEOJO02HV3 type=write size=1978358 2024-03-04T17:50:18.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.100+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=MAN9T5ZUJJCP type=del 2024-03-04T17:50:18.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.108+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~4IPJ3XSMQQNR.0 size=500311 2024-03-04T17:50:18.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.108+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=4IPJ3XSMQQNR tag=N5HR1K2TEJ1Q type=write size=1548887 2024-03-04T17:50:18.113 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.112+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~2EC2C857EB1A.0 size=1048576 2024-03-04T17:50:18.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.116+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~2EC2C857EB1A.1 size=1048576 2024-03-04T17:50:18.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.116+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~2EC2C857EB1A.2 size=605085 2024-03-04T17:50:18.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.124+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=2EC2C857EB1A tag=O2QLKC2T0BUU type=write size=3750813 2024-03-04T17:50:18.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.128+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=ZF8L1IZAXDN4 type=del size=2013176 2024-03-04T17:50:18.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.132+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~014V6DHGC6R2.0 size=207430 2024-03-04T17:50:18.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.140+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=014V6DHGC6R2 tag=RT0SU3MJB5P6 type=write size=1256006 2024-03-04T17:50:18.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.148+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~S6E4QW2Y0I9A.0 size=1048576 2024-03-04T17:50:18.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.148+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~S6E4QW2Y0I9A.1 size=1048576 2024-03-04T17:50:18.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.152+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~S6E4QW2Y0I9A.2 size=254082 2024-03-04T17:50:18.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.152+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=S6E4QW2Y0I9A tag=NEP9SA9UVERS type=write size=3399810 2024-03-04T17:50:18.159 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.156+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=DIE7BTSTSNGG type=del 2024-03-04T17:50:18.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.164+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~93ZCPQU3NIJR.0 size=900266 2024-03-04T17:50:18.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.164+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=93ZCPQU3NIJR tag=CGE2PTBTY413 type=write size=1948842 2024-03-04T17:50:18.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.168+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=Q4NDD4ZFW0VB type=del 2024-03-04T17:50:18.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.176+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=G1W7TD4W80P7 type=write size=827067 2024-03-04T17:50:18.180 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.180+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=K5ZHXZV3USRP type=del 2024-03-04T17:50:18.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.188+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~7Z00HOYW4OJE.0 size=1048576 2024-03-04T17:50:18.191 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.188+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~7Z00HOYW4OJE.1 size=1008809 2024-03-04T17:50:18.191 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.188+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=7Z00HOYW4OJE tag=SNRIF0V24HBG type=write size=3105961 2024-03-04T17:50:18.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.196+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=2ATMLAMWEFT2 type=del 2024-03-04T17:50:18.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.200+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~KZYU2QQVUYFQ.0 size=1048576 2024-03-04T17:50:18.206 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.204+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~KZYU2QQVUYFQ.1 size=345458 2024-03-04T17:50:18.206 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.204+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=KZYU2QQVUYFQ tag=QR09OSSW58AP type=write size=2442610 2024-03-04T17:50:18.210 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.208+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=I4DIDNW1VTUX type=del 2024-03-04T17:50:18.217 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.216+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~2IRHO12QMKG4.0 size=1048576 2024-03-04T17:50:18.220 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.220+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~2IRHO12QMKG4.1 size=1007411 2024-03-04T17:50:18.220 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.220+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=2IRHO12QMKG4 tag=9IA0J3RPTM0P type=write size=3104563 2024-03-04T17:50:18.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.224+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~CEO48A45W17K.0 size=1048576 2024-03-04T17:50:18.227 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.224+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~CEO48A45W17K.1 size=1048576 2024-03-04T17:50:18.229 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.228+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~CEO48A45W17K.2 size=451487 2024-03-04T17:50:18.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.236+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=CEO48A45W17K tag=OZ8JVT062A0X type=write size=3597215 2024-03-04T17:50:18.240 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.240+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=AEO7XS2EBM7T type=del size=603218 2024-03-04T17:50:18.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.244+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=QR5UEIFRMPTW type=del 2024-03-04T17:50:18.252 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.252+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~VZOQP1TBVKNB.0 size=1048576 2024-03-04T17:50:18.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.256+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~VZOQP1TBVKNB.1 size=440951 2024-03-04T17:50:18.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.256+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=VZOQP1TBVKNB tag=GGSE9N0SH8TW type=write size=2538103 2024-03-04T17:50:18.259 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.256+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~Z212GLWQH7FX.0 size=1048576 2024-03-04T17:50:18.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.260+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~Z212GLWQH7FX.1 size=933782 2024-03-04T17:50:18.268 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.268+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=Z212GLWQH7FX tag=ZIV77TXY9F3U type=write size=3030934 2024-03-04T17:50:18.274 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.272+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~CUNWQU8KK11B.0 size=1048576 2024-03-04T17:50:18.277 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.276+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~CUNWQU8KK11B.1 size=1048576 2024-03-04T17:50:18.280 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.280+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~CUNWQU8KK11B.2 size=562095 2024-03-04T17:50:18.280 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.280+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=CUNWQU8KK11B tag=J7U2ELY3XP08 type=write size=3707823 2024-03-04T17:50:18.284 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.284+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~4AA6ZO7XMN8M.0 size=1048576 2024-03-04T17:50:18.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.284+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~4AA6ZO7XMN8M.1 size=125522 2024-03-04T17:50:18.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.292+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=4AA6ZO7XMN8M tag=V493JERBPD73 type=write size=2222674 2024-03-04T17:50:18.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.296+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~FG6OJ1L6UWI3.0 size=576796 2024-03-04T17:50:18.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.296+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=FG6OJ1L6UWI3 tag=6J2GXL6342O1 type=write size=1625372 2024-03-04T17:50:18.303 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.304+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=MVSV4JQ26OAM type=del 2024-03-04T17:50:18.310 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.308+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~Y2K0QTH2O69P.0 size=1048576 2024-03-04T17:50:18.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.312+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~Y2K0QTH2O69P.1 size=1048576 2024-03-04T17:50:18.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.316+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~Y2K0QTH2O69P.2 size=558188 2024-03-04T17:50:18.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.316+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=Y2K0QTH2O69P tag=5R1V3YKIEANV type=write size=3703916 2024-03-04T17:50:18.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.320+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=UJLVGNPTD0LI type=del 2024-03-04T17:50:18.326 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.324+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~AC2XQZ6NLXIB.0 size=1048576 2024-03-04T17:50:18.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.328+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~AC2XQZ6NLXIB.1 size=503168 2024-03-04T17:50:18.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.328+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=AC2XQZ6NLXIB tag=D7CAO5MZQC4M type=write size=2600320 2024-03-04T17:50:18.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.332+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=Q5RQMTUSBNKW type=del 2024-03-04T17:50:18.337 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.336+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=Y4RNKKAPA9ZH type=del size=2992973 2024-03-04T17:50:18.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.340+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~KDKGS5AR36KJ.0 size=929782 2024-03-04T17:50:18.348 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.348+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=KDKGS5AR36KJ tag=PLWEOJO02HV3 type=write size=1978358 2024-03-04T17:50:18.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.356+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~VKYGAXWLX746.0 size=337122 2024-03-04T17:50:18.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.356+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=VKYGAXWLX746 tag=QNBK1SKIB32C type=write size=1385698 2024-03-04T17:50:18.361 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.360+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~4IPJ3XSMQQNR.0 size=500311 2024-03-04T17:50:18.367 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.368+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=4IPJ3XSMQQNR tag=N5HR1K2TEJ1Q type=write size=1548887 2024-03-04T17:50:18.371 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.372+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=QKU39E6SH7F4 type=del size=3985225 2024-03-04T17:50:18.375 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.376+0000 7fe960f568c0 1 < canceled operation key=P28N6U79ZUV2 tag=ZF8L1IZAXDN4 type=del size=2013176 2024-03-04T17:50:18.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.376+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=8LWKRE89GMJM type=write size=585691 2024-03-04T17:50:18.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.384+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~S6E4QW2Y0I9A.0 size=1048576 2024-03-04T17:50:18.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.384+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~S6E4QW2Y0I9A.1 size=1048576 2024-03-04T17:50:18.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.388+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~S6E4QW2Y0I9A.2 size=254082 2024-03-04T17:50:18.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.396+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=S6E4QW2Y0I9A tag=NEP9SA9UVERS type=write size=3399810 2024-03-04T17:50:18.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.400+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~TQD3GD7TRCXL.0 size=822085 2024-03-04T17:50:18.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.400+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=TQD3GD7TRCXL tag=MJDO3OB28RP4 type=write size=1870661 2024-03-04T17:50:18.406 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.404+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~93ZCPQU3NIJR.0 size=900266 2024-03-04T17:50:18.412 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.412+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=93ZCPQU3NIJR tag=CGE2PTBTY413 type=write size=1948842 2024-03-04T17:50:18.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.416+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~RGCA914SNXYZ.0 size=1048576 2024-03-04T17:50:18.422 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.420+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~RGCA914SNXYZ.1 size=542052 2024-03-04T17:50:18.422 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.420+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=RGCA914SNXYZ tag=J9S85TXK71TH type=write size=2639204 2024-03-04T17:50:18.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.424+0000 7fe960f568c0 1 < completed write operation key=7Z7BPZ8XGBHZ tag=G1W7TD4W80P7 type=write size=827067 2024-03-04T17:50:18.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.432+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~S2QGAJI4JZTE.0 size=1048576 2024-03-04T17:50:18.436 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.436+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~S2QGAJI4JZTE.1 size=1048576 2024-03-04T17:50:18.442 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.440+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~S2QGAJI4JZTE.2 size=604849 2024-03-04T17:50:18.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.440+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=S2QGAJI4JZTE tag=290YG4UAVIMM type=write size=3750577 2024-03-04T17:50:18.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.444+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~7Z00HOYW4OJE.0 size=1048576 2024-03-04T17:50:18.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.448+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~7Z00HOYW4OJE.1 size=1008809 2024-03-04T17:50:18.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.456+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=7Z00HOYW4OJE tag=SNRIF0V24HBG type=write size=3105961 2024-03-04T17:50:18.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.460+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~MJZJ4K7D06H5.0 size=662368 2024-03-04T17:50:18.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.460+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=MJZJ4K7D06H5 tag=17ZN99DYAFXT type=write size=1710944 2024-03-04T17:50:18.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.464+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~KZYU2QQVUYFQ.0 size=1048576 2024-03-04T17:50:18.469 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.468+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~KZYU2QQVUYFQ.1 size=345458 2024-03-04T17:50:18.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.476+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=KZYU2QQVUYFQ tag=QR09OSSW58AP type=write size=2442610 2024-03-04T17:50:18.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.476+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=STCB5F6XMD5J type=del size=1861497 2024-03-04T17:50:18.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.480+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~2IRHO12QMKG4.0 size=1048576 2024-03-04T17:50:18.486 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.484+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~2IRHO12QMKG4.1 size=1007411 2024-03-04T17:50:18.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.492+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=2IRHO12QMKG4 tag=9IA0J3RPTM0P type=write size=3104563 2024-03-04T17:50:18.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.492+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=UGRNJY4I627M type=del size=2257419 2024-03-04T17:50:18.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.496+0000 7fe960f568c0 1 < canceled operation key=ASC735ZCBLCG tag=AEO7XS2EBM7T type=del size=603218 2024-03-04T17:50:18.502 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.500+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=J2NJKKEZ5P1S type=del size=3891857 2024-03-04T17:50:18.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.504+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~VZOQP1TBVKNB.0 size=1048576 2024-03-04T17:50:18.509 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.508+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~VZOQP1TBVKNB.1 size=440951 2024-03-04T17:50:18.515 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.516+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=VZOQP1TBVKNB tag=GGSE9N0SH8TW type=write size=2538103 2024-03-04T17:50:18.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.520+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~EUTS8E8OOT0H.0 size=1048576 2024-03-04T17:50:18.526 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.524+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~EUTS8E8OOT0H.1 size=1048576 2024-03-04T17:50:18.529 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.528+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~EUTS8E8OOT0H.2 size=384224 2024-03-04T17:50:18.529 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.528+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=EUTS8E8OOT0H tag=LSOJB2IHSXKK type=write size=3529952 2024-03-04T17:50:18.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.532+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~CUNWQU8KK11B.0 size=1048576 2024-03-04T17:50:18.536 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.536+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~CUNWQU8KK11B.1 size=1048576 2024-03-04T17:50:18.539 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.536+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~CUNWQU8KK11B.2 size=562095 2024-03-04T17:50:18.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.544+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=CUNWQU8KK11B tag=J7U2ELY3XP08 type=write size=3707823 2024-03-04T17:50:18.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.548+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=S97U1MIEFMTF type=write size=795691 2024-03-04T17:50:18.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.552+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~FG6OJ1L6UWI3.0 size=576796 2024-03-04T17:50:18.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.556+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=FG6OJ1L6UWI3 tag=6J2GXL6342O1 type=write size=1625372 2024-03-04T17:50:18.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.560+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=XZ7XQK5FQULC type=del size=3513339 2024-03-04T17:50:18.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.564+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~Y2K0QTH2O69P.0 size=1048576 2024-03-04T17:50:18.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.568+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~Y2K0QTH2O69P.1 size=1048576 2024-03-04T17:50:18.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.572+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~Y2K0QTH2O69P.2 size=558188 2024-03-04T17:50:18.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.576+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=Y2K0QTH2O69P tag=5R1V3YKIEANV type=write size=3703916 2024-03-04T17:50:18.581 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.580+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=SMB3UTFTIEDK type=del size=3729452 2024-03-04T17:50:18.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.584+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~AC2XQZ6NLXIB.0 size=1048576 2024-03-04T17:50:18.588 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.588+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~AC2XQZ6NLXIB.1 size=503168 2024-03-04T17:50:18.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.592+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=AC2XQZ6NLXIB tag=D7CAO5MZQC4M type=write size=2600320 2024-03-04T17:50:18.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.596+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=4PYOO8N87P3K type=del size=1026650 2024-03-04T17:50:18.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.600+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=Y4RNKKAPA9ZH type=del 2024-03-04T17:50:18.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.604+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=6EGFOL4XCQD7 type=write size=691948 2024-03-04T17:50:18.610 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.608+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~VKYGAXWLX746.0 size=337122 2024-03-04T17:50:18.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.616+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=VKYGAXWLX746 tag=QNBK1SKIB32C type=write size=1385698 2024-03-04T17:50:18.624 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.624+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~9D5PJPSFOHLA.0 size=1048576 2024-03-04T17:50:18.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.624+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~9D5PJPSFOHLA.1 size=1048576 2024-03-04T17:50:18.630 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.628+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~9D5PJPSFOHLA.2 size=907937 2024-03-04T17:50:18.630 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.628+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=9D5PJPSFOHLA tag=CQ584FTCPBYR type=write size=4053665 2024-03-04T17:50:18.633 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.632+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=QKU39E6SH7F4 type=del 2024-03-04T17:50:18.637 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.636+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=VHSFZ7Z7LAO2 type=del size=2305765 2024-03-04T17:50:18.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.640+0000 7fe960f568c0 1 < completed write operation key=UEVMIPQ9FR81 tag=8LWKRE89GMJM type=write size=585691 2024-03-04T17:50:18.644 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.644+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=I0XBB0W9WNXW type=del size=468934 2024-03-04T17:50:18.648 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.648+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~TQD3GD7TRCXL.0 size=822085 2024-03-04T17:50:18.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.652+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=TQD3GD7TRCXL tag=MJDO3OB28RP4 type=write size=1870661 2024-03-04T17:50:18.658 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.656+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=LZYNE0JBHPTW type=del size=1158445 2024-03-04T17:50:18.662 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.660+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~RGCA914SNXYZ.0 size=1048576 2024-03-04T17:50:18.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.664+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~RGCA914SNXYZ.1 size=542052 2024-03-04T17:50:18.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.668+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=RGCA914SNXYZ tag=J9S85TXK71TH type=write size=2639204 2024-03-04T17:50:18.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.672+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=KTAP7AWHWKJH type=del size=1914808 2024-03-04T17:50:18.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.676+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~S2QGAJI4JZTE.0 size=1048576 2024-03-04T17:50:18.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.680+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~S2QGAJI4JZTE.1 size=1048576 2024-03-04T17:50:18.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.684+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~S2QGAJI4JZTE.2 size=604849 2024-03-04T17:50:18.691 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.688+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=S2QGAJI4JZTE tag=290YG4UAVIMM type=write size=3750577 2024-03-04T17:50:18.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.692+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=IWZAXEUKI6SX type=del size=3206215 2024-03-04T17:50:18.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.696+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~MJZJ4K7D06H5.0 size=662368 2024-03-04T17:50:18.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.704+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=MJZJ4K7D06H5 tag=17ZN99DYAFXT type=write size=1710944 2024-03-04T17:50:18.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.708+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=9CW7C3PHRUTV type=del size=2008739 2024-03-04T17:50:18.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.712+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=STCB5F6XMD5J type=del 2024-03-04T17:50:18.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.716+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=G4X5NGW13WEG type=write size=650555 2024-03-04T17:50:18.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.720+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=UGRNJY4I627M type=del 2024-03-04T17:50:18.723 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.724+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=W1XRA4PTH2M8 type=write size=118222 2024-03-04T17:50:18.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.728+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=J2NJKKEZ5P1S type=del 2024-03-04T17:50:18.735 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.732+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~LVALB3DVVW9W.0 size=1048576 2024-03-04T17:50:18.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.736+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~LVALB3DVVW9W.1 size=1048576 2024-03-04T17:50:18.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.740+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~LVALB3DVVW9W.2 size=648620 2024-03-04T17:50:18.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.740+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=LVALB3DVVW9W tag=KXD33X07J62S type=write size=3794348 2024-03-04T17:50:18.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.744+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~EUTS8E8OOT0H.0 size=1048576 2024-03-04T17:50:18.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.748+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~EUTS8E8OOT0H.1 size=1048576 2024-03-04T17:50:18.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.752+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~EUTS8E8OOT0H.2 size=384224 2024-03-04T17:50:18.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.756+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=EUTS8E8OOT0H tag=LSOJB2IHSXKK type=write size=3529952 2024-03-04T17:50:18.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.764+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~YHRW1VUACAR1.0 size=1048576 2024-03-04T17:50:18.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.768+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~YHRW1VUACAR1.1 size=19515 2024-03-04T17:50:18.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.768+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=YHRW1VUACAR1 tag=MV39VBC4T9BF type=write size=2116667 2024-03-04T17:50:18.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.776+0000 7fe960f568c0 1 < suggested update operation key=RII14ZGX2EL1 tag=S97U1MIEFMTF type=write size=795691 2024-03-04T17:50:18.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.788+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~K1HYB725M58V.0 size=1017426 2024-03-04T17:50:18.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.788+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=K1HYB725M58V tag=69OJVQ3HLNTW type=write size=2066002 2024-03-04T17:50:18.795 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.792+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=XZ7XQK5FQULC type=del 2024-03-04T17:50:18.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.800+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~S2KHQB0BYRVG.0 size=1048576 2024-03-04T17:50:18.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.804+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~S2KHQB0BYRVG.1 size=1048576 2024-03-04T17:50:18.808 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.808+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~S2KHQB0BYRVG.2 size=889248 2024-03-04T17:50:18.808 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.808+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=S2KHQB0BYRVG tag=Q3VZRWODHGQ3 type=write size=4034976 2024-03-04T17:50:18.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.812+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=SMB3UTFTIEDK type=del 2024-03-04T17:50:18.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.816+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=DTD9JY4ZBN9S type=del size=246974 2024-03-04T17:50:18.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.820+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=4PYOO8N87P3K type=del 2024-03-04T17:50:18.826 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.824+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~V8UWPD9SU3KN.0 size=180862 2024-03-04T17:50:18.826 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.824+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=V8UWPD9SU3KN tag=AJV9405W2222 type=write size=1229438 2024-03-04T17:50:18.831 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.828+0000 7fe960f568c0 1 < completed write operation key=FSWBX523X9V5 tag=6EGFOL4XCQD7 type=write size=691948 2024-03-04T17:50:18.835 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.832+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=4DS2H9Q8TIT4 type=write size=164453 2024-03-04T17:50:18.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.840+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~9D5PJPSFOHLA.0 size=1048576 2024-03-04T17:50:18.842 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.840+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~9D5PJPSFOHLA.1 size=1048576 2024-03-04T17:50:18.845 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.844+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~9D5PJPSFOHLA.2 size=907937 2024-03-04T17:50:18.851 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.848+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=9D5PJPSFOHLA tag=CQ584FTCPBYR type=write size=4053665 2024-03-04T17:50:18.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.852+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=3T8NYZODK6PM type=del size=2626311 2024-03-04T17:50:18.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.856+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=VHSFZ7Z7LAO2 type=del 2024-03-04T17:50:18.864 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.864+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~35KL1H90F362.0 size=1048576 2024-03-04T17:50:18.867 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.864+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~35KL1H90F362.1 size=1048576 2024-03-04T17:50:18.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.868+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~35KL1H90F362.2 size=560202 2024-03-04T17:50:18.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.868+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=35KL1H90F362 tag=5OL3SQKOQ67S type=write size=3705930 2024-03-04T17:50:18.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.872+0000 7fe960f568c0 -1 < failed to complete operation key=RII14ZGX2EL1 tag=I0XBB0W9WNXW type=del size=468934: (22) Invalid argument 2024-03-04T17:50:18.880 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.880+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~0DR6FUHVMC4V.0 size=1048576 2024-03-04T17:50:18.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.884+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~0DR6FUHVMC4V.1 size=349387 2024-03-04T17:50:18.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.884+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=0DR6FUHVMC4V tag=2N362484Q49F type=write size=2446539 2024-03-04T17:50:18.887 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.884+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=LZYNE0JBHPTW type=del 2024-03-04T17:50:18.894 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.892+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~L0LP6LTQFE85.0 size=1048576 2024-03-04T17:50:18.896 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.896+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~L0LP6LTQFE85.1 size=1048576 2024-03-04T17:50:18.899 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.896+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~L0LP6LTQFE85.2 size=706261 2024-03-04T17:50:18.899 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.896+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=L0LP6LTQFE85 tag=N7X6WIJQKK51 type=write size=3851989 2024-03-04T17:50:18.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.908+0000 7fe960f568c0 1 < suggested update operation key=QGLN27WH1N2O tag=KTAP7AWHWKJH type=del size=1914808 2024-03-04T17:50:18.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.912+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~VSO8V18C0T3L.0 size=888729 2024-03-04T17:50:18.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.912+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=VSO8V18C0T3L tag=HB2JKGNRI5GB type=write size=1937305 2024-03-04T17:50:18.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.916+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=IWZAXEUKI6SX type=del 2024-03-04T17:50:18.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.920+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=UQP3ILN1CVBB type=del size=3189495 2024-03-04T17:50:18.925 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.924+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=9CW7C3PHRUTV type=del 2024-03-04T17:50:18.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.932+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~NXIBF0XTMCB7.0 size=1048576 2024-03-04T17:50:18.934 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.932+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~NXIBF0XTMCB7.1 size=1048576 2024-03-04T17:50:18.937 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.936+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~NXIBF0XTMCB7.2 size=548911 2024-03-04T17:50:18.937 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.936+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=NXIBF0XTMCB7 tag=XQKO2QRH9TPN type=write size=3694639 2024-03-04T17:50:18.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.940+0000 7fe960f568c0 1 < completed write operation key=GZ4RRIR5CC26 tag=G4X5NGW13WEG type=write size=650555 2024-03-04T17:50:18.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.944+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=8355777BPL0Z type=del size=1734243 2024-03-04T17:50:18.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.948+0000 7fe960f568c0 1 < completed write operation key=R6ZDVOUE9NXJ tag=W1XRA4PTH2M8 type=write size=118222 2024-03-04T17:50:18.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.952+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=MHHWOM48M7Q9 type=del size=1270451 2024-03-04T17:50:18.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.956+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~LVALB3DVVW9W.0 size=1048576 2024-03-04T17:50:18.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.956+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~LVALB3DVVW9W.1 size=1048576 2024-03-04T17:50:18.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.960+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~LVALB3DVVW9W.2 size=648620 2024-03-04T17:50:18.969 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.968+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=LVALB3DVVW9W tag=KXD33X07J62S type=write size=3794348 2024-03-04T17:50:18.972 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.972+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=MRY2W23RKEYK type=del size=3246026 2024-03-04T17:50:18.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.976+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~YHRW1VUACAR1.0 size=1048576 2024-03-04T17:50:18.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.980+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~YHRW1VUACAR1.1 size=19515 2024-03-04T17:50:18.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.984+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=YHRW1VUACAR1 tag=MV39VBC4T9BF type=write size=2116667 2024-03-04T17:50:18.991 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.988+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=0IFIO7KTK1EA type=del size=1387329 2024-03-04T17:50:18.995 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:18.996+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~K1HYB725M58V.0 size=1017426 2024-03-04T17:50:19.002 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.000+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=K1HYB725M58V tag=69OJVQ3HLNTW type=write size=2066002 2024-03-04T17:50:19.005 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.004+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=J5M03WUDFL3T type=del size=2273301 2024-03-04T17:50:19.009 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.008+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~S2KHQB0BYRVG.0 size=1048576 2024-03-04T17:50:19.012 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.012+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~S2KHQB0BYRVG.1 size=1048576 2024-03-04T17:50:19.012 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.012+0000 7fe960f568c0 1 < canceled part key=_multipart_WNXVI6K5MU3R.2~S2KHQB0BYRVG.2 size=889248 2024-03-04T17:50:19.018 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.016+0000 7fe960f568c0 1 < canceled multipart upload key=WNXVI6K5MU3R upload=S2KHQB0BYRVG tag=Q3VZRWODHGQ3 type=write size=4034976 2024-03-04T17:50:19.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.024+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~14LEVLHNWBAW.0 size=1048576 2024-03-04T17:50:19.028 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.028+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~14LEVLHNWBAW.1 size=963402 2024-03-04T17:50:19.028 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.028+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=14LEVLHNWBAW tag=AAXO1JVD3YBI type=write size=3060554 2024-03-04T17:50:19.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.032+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=DTD9JY4ZBN9S type=del 2024-03-04T17:50:19.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.036+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=KUFAFPNC125J type=del size=1355624 2024-03-04T17:50:19.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.036+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~V8UWPD9SU3KN.0 size=180862 2024-03-04T17:50:19.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.044+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=V8UWPD9SU3KN tag=AJV9405W2222 type=write size=1229438 2024-03-04T17:50:19.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.048+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=O8UM4NVDUMM3 type=write size=317037 2024-03-04T17:50:19.053 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.052+0000 7fe960f568c0 1 < completed write operation key=ASC735ZCBLCG tag=4DS2H9Q8TIT4 type=write size=164453 2024-03-04T17:50:19.056 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.056+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=LYQKHMT1H9DJ type=del size=823235 2024-03-04T17:50:19.060 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.060+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=3T8NYZODK6PM type=del 2024-03-04T17:50:19.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.064+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=DTG06AVGK6RJ type=del size=438867 2024-03-04T17:50:19.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.068+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~35KL1H90F362.0 size=1048576 2024-03-04T17:50:19.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.068+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~35KL1H90F362.1 size=1048576 2024-03-04T17:50:19.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.072+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~35KL1H90F362.2 size=560202 2024-03-04T17:50:19.080 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.080+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=35KL1H90F362 tag=5OL3SQKOQ67S type=write size=3705930 2024-03-04T17:50:19.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.084+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=VW9LN7AW37EH type=del size=1860514 2024-03-04T17:50:19.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.088+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~0DR6FUHVMC4V.0 size=1048576 2024-03-04T17:50:19.091 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.088+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~0DR6FUHVMC4V.1 size=349387 2024-03-04T17:50:19.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.096+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=0DR6FUHVMC4V tag=2N362484Q49F type=write size=2446539 2024-03-04T17:50:19.103 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.104+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~Y45K6W1XHA88.0 size=1048576 2024-03-04T17:50:19.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.104+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~Y45K6W1XHA88.1 size=1048576 2024-03-04T17:50:19.109 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.108+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~Y45K6W1XHA88.2 size=645610 2024-03-04T17:50:19.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.108+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=Y45K6W1XHA88 tag=4MGHRF7HG1M4 type=write size=3791338 2024-03-04T17:50:19.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.112+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~L0LP6LTQFE85.0 size=1048576 2024-03-04T17:50:19.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.112+0000 7fe960f568c0 1 < canceled part key=_multipart_2KZ4IFW54C5B.2~L0LP6LTQFE85.1 size=1048576 2024-03-04T17:50:19.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.116+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~L0LP6LTQFE85.2 size=706261 2024-03-04T17:50:19.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.124+0000 7fe960f568c0 1 < canceled multipart upload key=2KZ4IFW54C5B upload=L0LP6LTQFE85 tag=N7X6WIJQKK51 type=write size=3851989 2024-03-04T17:50:19.127 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.128+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=3EAWYU5ZK33H type=del size=2704240 2024-03-04T17:50:19.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.132+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~VSO8V18C0T3L.0 size=888729 2024-03-04T17:50:19.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.140+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=VSO8V18C0T3L tag=HB2JKGNRI5GB type=write size=1937305 2024-03-04T17:50:19.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.144+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~FKDX4RAPNUW5.0 size=1048576 2024-03-04T17:50:19.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.148+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~FKDX4RAPNUW5.1 size=585060 2024-03-04T17:50:19.150 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.148+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=FKDX4RAPNUW5 tag=XGU2JAX91Q52 type=write size=2682212 2024-03-04T17:50:19.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.152+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=UQP3ILN1CVBB type=del 2024-03-04T17:50:19.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.160+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~XRCJCM4B00XH.0 size=1048576 2024-03-04T17:50:19.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.164+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~XRCJCM4B00XH.1 size=1048576 2024-03-04T17:50:19.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.168+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~XRCJCM4B00XH.2 size=201955 2024-03-04T17:50:19.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.168+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=XRCJCM4B00XH tag=DTKJB32BXVRM type=write size=3347683 2024-03-04T17:50:19.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.172+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~NXIBF0XTMCB7.0 size=1048576 2024-03-04T17:50:19.174 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.172+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~NXIBF0XTMCB7.1 size=1048576 2024-03-04T17:50:19.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.176+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~NXIBF0XTMCB7.2 size=548911 2024-03-04T17:50:19.183 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.184+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=NXIBF0XTMCB7 tag=XQKO2QRH9TPN type=write size=3694639 2024-03-04T17:50:19.188 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.188+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=3QMZI6L27CWM type=write size=100373 2024-03-04T17:50:19.192 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.192+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=8355777BPL0Z type=del 2024-03-04T17:50:19.199 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.200+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~83TVBSQ6VWJM.0 size=362280 2024-03-04T17:50:19.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.200+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=83TVBSQ6VWJM tag=L6NT19XVY1QL type=write size=1410856 2024-03-04T17:50:19.203 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.200+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=MHHWOM48M7Q9 type=del 2024-03-04T17:50:19.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.208+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~PR9257HWJ1R5.0 size=1048576 2024-03-04T17:50:19.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.212+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~PR9257HWJ1R5.1 size=176280 2024-03-04T17:50:19.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.212+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=PR9257HWJ1R5 tag=LKV2JX8VBN95 type=write size=2273432 2024-03-04T17:50:19.216 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.216+0000 7fe960f568c0 1 < canceled operation key=R6ZDVOUE9NXJ tag=MRY2W23RKEYK type=del size=3246026 2024-03-04T17:50:19.219 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.220+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=FTB95FD45Q21 type=write size=344845 2024-03-04T17:50:19.227 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.224+0000 7fe960f568c0 1 < suggested update operation key=KYSLL05AUHAC tag=0IFIO7KTK1EA type=del size=1387329 2024-03-04T17:50:19.230 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.228+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=LF10TNFYP2T8 type=write size=973528 2024-03-04T17:50:19.234 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.232+0000 7fe960f568c0 1 < completed delete operation key=U2G6YEVD5506 tag=J5M03WUDFL3T type=del 2024-03-04T17:50:19.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.236+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=EMIG42QER2U1 type=del size=2438657 2024-03-04T17:50:19.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.240+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~14LEVLHNWBAW.0 size=1048576 2024-03-04T17:50:19.244 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.244+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~14LEVLHNWBAW.1 size=963402 2024-03-04T17:50:19.250 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.248+0000 7fe960f568c0 1 < completed multipart upload key=ASC735ZCBLCG upload=14LEVLHNWBAW tag=AAXO1JVD3YBI type=write size=3060554 2024-03-04T17:50:19.257 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.256+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~4W003WQPZH8N.0 size=1048576 2024-03-04T17:50:19.261 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.260+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~4W003WQPZH8N.1 size=704352 2024-03-04T17:50:19.261 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.260+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=4W003WQPZH8N tag=2XDRU7IEDSGT type=write size=2801504 2024-03-04T17:50:19.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.264+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=KUFAFPNC125J type=del 2024-03-04T17:50:19.269 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.268+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=3W1M4IH8A39L type=del size=2204541 2024-03-04T17:50:19.273 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.272+0000 7fe960f568c0 1 < completed write operation key=7Z7BPZ8XGBHZ tag=O8UM4NVDUMM3 type=write size=317037 2024-03-04T17:50:19.276 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.276+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=R51AGDW4G24W type=del size=2300364 2024-03-04T17:50:19.280 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.280+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=LYQKHMT1H9DJ type=del 2024-03-04T17:50:19.284 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.284+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=L8NLI4XDA42M type=del size=330914 2024-03-04T17:50:19.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.288+0000 7fe960f568c0 -1 < failed to complete operation key=KYSLL05AUHAC tag=DTG06AVGK6RJ type=del size=438867: (22) Invalid argument 2024-03-04T17:50:19.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.292+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=TDPMVFV35CXT type=del size=2165085 2024-03-04T17:50:19.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.296+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=VW9LN7AW37EH type=del 2024-03-04T17:50:19.300 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.300+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=HS41XFAVJI84 type=del size=3072300 2024-03-04T17:50:19.304 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.304+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~Y45K6W1XHA88.0 size=1048576 2024-03-04T17:50:19.307 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.304+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~Y45K6W1XHA88.1 size=1048576 2024-03-04T17:50:19.307 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.304+0000 7fe960f568c0 1 < canceled part key=_multipart_MIZRUP4MEA0J.2~Y45K6W1XHA88.2 size=645610 2024-03-04T17:50:19.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.312+0000 7fe960f568c0 1 < canceled multipart upload key=MIZRUP4MEA0J upload=Y45K6W1XHA88 tag=4MGHRF7HG1M4 type=write size=3791338 2024-03-04T17:50:19.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.320+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~T2MASI1PPPE2.0 size=1048576 2024-03-04T17:50:19.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.324+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~T2MASI1PPPE2.1 size=456935 2024-03-04T17:50:19.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.324+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=T2MASI1PPPE2 tag=Q0FSULNG5RXB type=write size=2554087 2024-03-04T17:50:19.327 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.328+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=3EAWYU5ZK33H type=del 2024-03-04T17:50:19.334 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.332+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~X1UCB46O1XQN.0 size=1048576 2024-03-04T17:50:19.337 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.336+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~X1UCB46O1XQN.1 size=1048576 2024-03-04T17:50:19.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.340+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~X1UCB46O1XQN.2 size=864891 2024-03-04T17:50:19.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.340+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=X1UCB46O1XQN tag=B4L3LTISR4UH type=write size=4010619 2024-03-04T17:50:19.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.344+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~FKDX4RAPNUW5.0 size=1048576 2024-03-04T17:50:19.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.344+0000 7fe960f568c0 1 < canceled part key=_multipart_3X4719GHK3BI.2~FKDX4RAPNUW5.1 size=585060 2024-03-04T17:50:19.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.348+0000 7fe960f568c0 1 < canceled multipart upload key=3X4719GHK3BI upload=FKDX4RAPNUW5 tag=XGU2JAX91Q52 type=write size=2682212 2024-03-04T17:50:19.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.352+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=EJG6BGNJMR16 type=del size=2319458 2024-03-04T17:50:19.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.356+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~XRCJCM4B00XH.0 size=1048576 2024-03-04T17:50:19.358 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.356+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~XRCJCM4B00XH.1 size=1048576 2024-03-04T17:50:19.361 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.360+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~XRCJCM4B00XH.2 size=201955 2024-03-04T17:50:19.367 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.364+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=XRCJCM4B00XH tag=DTKJB32BXVRM type=write size=3347683 2024-03-04T17:50:19.373 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.372+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~GZC993J1CP9O.0 size=84144 2024-03-04T17:50:19.373 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.372+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=GZC993J1CP9O tag=C7XFB58YWTCZ type=write size=1132720 2024-03-04T17:50:19.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.376+0000 7fe960f568c0 1 < canceled operation key=JJ7SM1Y73A4C tag=3QMZI6L27CWM type=write size=100373 2024-03-04T17:50:19.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.380+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~D5QPKLPSTOZV.0 size=1048576 2024-03-04T17:50:19.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.384+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~D5QPKLPSTOZV.1 size=486125 2024-03-04T17:50:19.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.384+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=D5QPKLPSTOZV tag=VPSOLQNSZGEG type=write size=2583277 2024-03-04T17:50:19.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.388+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~83TVBSQ6VWJM.0 size=362280 2024-03-04T17:50:19.394 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.392+0000 7fe960f568c0 1 < completed multipart upload key=U2G6YEVD5506 upload=83TVBSQ6VWJM tag=L6NT19XVY1QL type=write size=1410856 2024-03-04T17:50:19.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.396+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=N31DT23TCR86 type=write size=249086 2024-03-04T17:50:19.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.400+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~PR9257HWJ1R5.0 size=1048576 2024-03-04T17:50:19.404 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.404+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~PR9257HWJ1R5.1 size=176280 2024-03-04T17:50:19.410 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.408+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=PR9257HWJ1R5 tag=LKV2JX8VBN95 type=write size=2273432 2024-03-04T17:50:19.414 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.412+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=9MKY25AKQDMY type=del size=2342286 2024-03-04T17:50:19.418 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.416+0000 7fe960f568c0 1 < completed write operation key=NF4OIJWJ83DE tag=FTB95FD45Q21 type=write size=344845 2024-03-04T17:50:19.421 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.420+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=QX3GDZWGJ9JE type=del size=195594 2024-03-04T17:50:19.430 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.428+0000 7fe960f568c0 1 < suggested remove operation key=FSWBX523X9V5 tag=LF10TNFYP2T8 type=write 2024-03-04T17:50:19.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.432+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=K6RXE18A5N44 type=write size=836914 2024-03-04T17:50:19.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.436+0000 7fe960f568c0 1 < canceled operation key=R6ZDVOUE9NXJ tag=EMIG42QER2U1 type=del size=2438657 2024-03-04T17:50:19.448 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.444+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~FE2MTDVN1J6U.0 size=1048576 2024-03-04T17:50:19.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.448+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~FE2MTDVN1J6U.1 size=906734 2024-03-04T17:50:19.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.448+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=FE2MTDVN1J6U tag=UEQ3AB31WUC2 type=write size=3003886 2024-03-04T17:50:19.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.452+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~4W003WQPZH8N.0 size=1048576 2024-03-04T17:50:19.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.456+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~4W003WQPZH8N.1 size=704352 2024-03-04T17:50:19.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.464+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=4W003WQPZH8N tag=2XDRU7IEDSGT type=write size=2801504 2024-03-04T17:50:19.468 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.468+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=5VDF5ND21EGC type=del size=4123334 2024-03-04T17:50:19.472 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.472+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=3W1M4IH8A39L type=del 2024-03-04T17:50:19.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.476+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=FTNZ9DU3X9Z9 type=del size=223013 2024-03-04T17:50:19.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.476+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=R51AGDW4G24W type=del 2024-03-04T17:50:19.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.484+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=T5F0A8384JLO type=del size=3734357 2024-03-04T17:50:19.489 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.488+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=L8NLI4XDA42M type=del 2024-03-04T17:50:19.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.492+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~Z954LI3LCBM3.0 size=1048576 2024-03-04T17:50:19.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.496+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~Z954LI3LCBM3.1 size=1048576 2024-03-04T17:50:19.500 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.500+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~Z954LI3LCBM3.2 size=1031519 2024-03-04T17:50:19.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.500+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=Z954LI3LCBM3 tag=2XFYIE2ASLJ3 type=write size=4177247 2024-03-04T17:50:19.504 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.504+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=TDPMVFV35CXT type=del 2024-03-04T17:50:19.507 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.508+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=FAB01YB6ESRK type=del size=3119288 2024-03-04T17:50:19.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.508+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=HS41XFAVJI84 type=del 2024-03-04T17:50:19.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.516+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~O3S8CTXQTTLS.0 size=1048576 2024-03-04T17:50:19.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.520+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~O3S8CTXQTTLS.1 size=377857 2024-03-04T17:50:19.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.520+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=O3S8CTXQTTLS tag=7FAL4NL877IP type=write size=2475009 2024-03-04T17:50:19.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.524+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~T2MASI1PPPE2.0 size=1048576 2024-03-04T17:50:19.528 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.528+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~T2MASI1PPPE2.1 size=456935 2024-03-04T17:50:19.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.532+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=T2MASI1PPPE2 tag=Q0FSULNG5RXB type=write size=2554087 2024-03-04T17:50:19.537 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.536+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=3UJKZN00DLM2 type=write size=942784 2024-03-04T17:50:19.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.540+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~X1UCB46O1XQN.0 size=1048576 2024-03-04T17:50:19.546 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.544+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~X1UCB46O1XQN.1 size=1048576 2024-03-04T17:50:19.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.548+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~X1UCB46O1XQN.2 size=864891 2024-03-04T17:50:19.554 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.552+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=X1UCB46O1XQN tag=B4L3LTISR4UH type=write size=4010619 2024-03-04T17:50:19.561 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.560+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~PBZGTQGZGY2C.0 size=1048576 2024-03-04T17:50:19.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.564+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~PBZGTQGZGY2C.1 size=222973 2024-03-04T17:50:19.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.564+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=PBZGTQGZGY2C tag=ULWHQHVX2TEL type=write size=2320125 2024-03-04T17:50:19.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.568+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=EJG6BGNJMR16 type=del 2024-03-04T17:50:19.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.572+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=P92SZW9F6K6I type=del size=2483525 2024-03-04T17:50:19.576 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.576+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~GZC993J1CP9O.0 size=84144 2024-03-04T17:50:19.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.580+0000 7fe960f568c0 1 < completed multipart upload key=XSQ3GE2UHMZG upload=GZC993J1CP9O tag=C7XFB58YWTCZ type=write size=1132720 2024-03-04T17:50:19.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.584+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=P4ABV6TY2ZEX type=del size=488445 2024-03-04T17:50:19.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.588+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~D5QPKLPSTOZV.0 size=1048576 2024-03-04T17:50:19.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.592+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~D5QPKLPSTOZV.1 size=486125 2024-03-04T17:50:19.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.596+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=D5QPKLPSTOZV tag=VPSOLQNSZGEG type=write size=2583277 2024-03-04T17:50:19.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.600+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=QLJ01PDJG2HQ type=del size=594118 2024-03-04T17:50:19.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.604+0000 7fe960f568c0 1 < canceled operation key=QFF67JIW8Y8E tag=N31DT23TCR86 type=write size=249086 2024-03-04T17:50:19.610 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.608+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=AGT1PJ6ZXJUN type=del size=133870 2024-03-04T17:50:19.614 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.612+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=9MKY25AKQDMY type=del 2024-03-04T17:50:19.621 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.620+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~QC4RX2K0TE06.0 size=1048576 2024-03-04T17:50:19.623 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.624+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~QC4RX2K0TE06.1 size=482795 2024-03-04T17:50:19.623 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.624+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=QC4RX2K0TE06 tag=98688UOR6GC7 type=write size=2579947 2024-03-04T17:50:19.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.628+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=QX3GDZWGJ9JE type=del 2024-03-04T17:50:19.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.628+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=T1EQ49Z881WQ type=write size=73961 2024-03-04T17:50:19.635 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.632+0000 7fe960f568c0 1 < completed write operation key=FLYDZHOIWO6L tag=K6RXE18A5N44 type=write size=836914 2024-03-04T17:50:19.638 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.636+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=JAG4E91WB03T type=del size=1433686 2024-03-04T17:50:19.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.640+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~FE2MTDVN1J6U.0 size=1048576 2024-03-04T17:50:19.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.644+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~FE2MTDVN1J6U.1 size=906734 2024-03-04T17:50:19.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.652+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=FE2MTDVN1J6U tag=UEQ3AB31WUC2 type=write size=3003886 2024-03-04T17:50:19.655 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.656+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=SQMQTOJCGXAQ type=write size=1021641 2024-03-04T17:50:19.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.660+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=5VDF5ND21EGC type=del 2024-03-04T17:50:19.663 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.660+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=5MI39MDEVU09 type=del size=3067649 2024-03-04T17:50:19.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.664+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=FTNZ9DU3X9Z9 type=del 2024-03-04T17:50:19.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.672+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~OMIKVR6R8WU1.0 size=1048576 2024-03-04T17:50:19.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.676+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~OMIKVR6R8WU1.1 size=590927 2024-03-04T17:50:19.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.676+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=OMIKVR6R8WU1 tag=81LKXF3RWC0Z type=write size=2688079 2024-03-04T17:50:19.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.680+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=T5F0A8384JLO type=del 2024-03-04T17:50:19.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.684+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=O329XCYFBINL type=del size=3814920 2024-03-04T17:50:19.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.684+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~Z954LI3LCBM3.0 size=1048576 2024-03-04T17:50:19.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.688+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~Z954LI3LCBM3.1 size=1048576 2024-03-04T17:50:19.693 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.692+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~Z954LI3LCBM3.2 size=1031519 2024-03-04T17:50:19.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.696+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=Z954LI3LCBM3 tag=2XFYIE2ASLJ3 type=write size=4177247 2024-03-04T17:50:19.703 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.700+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=JSIC3IYC006R type=del size=2123651 2024-03-04T17:50:19.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.704+0000 7fe960f568c0 1 < canceled operation key=2MKQA5JYWU8L tag=FAB01YB6ESRK type=del size=3119288 2024-03-04T17:50:19.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.708+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=ZJU1Z10UIG3R type=del size=3280610 2024-03-04T17:50:19.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.712+0000 7fe960f568c0 1 < canceled part key=_multipart_MIZRUP4MEA0J.2~O3S8CTXQTTLS.0 size=1048576 2024-03-04T17:50:19.714 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.712+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~O3S8CTXQTTLS.1 size=377857 2024-03-04T17:50:19.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.720+0000 7fe960f568c0 1 < canceled multipart upload key=MIZRUP4MEA0J upload=O3S8CTXQTTLS tag=7FAL4NL877IP type=write size=2475009 2024-03-04T17:50:19.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.728+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~JMC0DNP71VHQ.0 size=1048576 2024-03-04T17:50:19.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.728+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~JMC0DNP71VHQ.1 size=405123 2024-03-04T17:50:19.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.728+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=JMC0DNP71VHQ tag=GT9SDKU5JFNT type=write size=2502275 2024-03-04T17:50:19.739 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.740+0000 7fe960f568c0 1 < suggested remove operation key=JT02WL9JYTNM tag=3UJKZN00DLM2 type=write 2024-03-04T17:50:19.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.744+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=BH4DXN974PVT type=del size=2973098 2024-03-04T17:50:19.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.744+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~PBZGTQGZGY2C.0 size=1048576 2024-03-04T17:50:19.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.748+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~PBZGTQGZGY2C.1 size=222973 2024-03-04T17:50:19.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.756+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=PBZGTQGZGY2C tag=ULWHQHVX2TEL type=write size=2320125 2024-03-04T17:50:19.760 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.760+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=Z48JN4XYANMM type=del size=376544 2024-03-04T17:50:19.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.764+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=P92SZW9F6K6I type=del 2024-03-04T17:50:19.767 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.768+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=VFE55VS8MNH9 type=write size=513713 2024-03-04T17:50:19.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.772+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=P4ABV6TY2ZEX type=del 2024-03-04T17:50:19.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.776+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~52HEK3BBPXH3.0 size=1048576 2024-03-04T17:50:19.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.780+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~52HEK3BBPXH3.1 size=434435 2024-03-04T17:50:19.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.780+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=52HEK3BBPXH3 tag=FYX4YMGBR6BK type=write size=2531587 2024-03-04T17:50:19.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.784+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=QLJ01PDJG2HQ type=del 2024-03-04T17:50:19.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.788+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=3XBHC7PPHX27 type=del size=101135 2024-03-04T17:50:19.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.792+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=AGT1PJ6ZXJUN type=del 2024-03-04T17:50:19.801 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.800+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~XF0HO9VUJ45N.0 size=1048576 2024-03-04T17:50:19.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.804+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~XF0HO9VUJ45N.1 size=1048576 2024-03-04T17:50:19.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.804+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~XF0HO9VUJ45N.2 size=790552 2024-03-04T17:50:19.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.804+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=XF0HO9VUJ45N tag=IUYTMYSTWH1J type=write size=3936280 2024-03-04T17:50:19.810 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.808+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~QC4RX2K0TE06.0 size=1048576 2024-03-04T17:50:19.814 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.812+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~QC4RX2K0TE06.1 size=482795 2024-03-04T17:50:19.820 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.820+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=QC4RX2K0TE06 tag=98688UOR6GC7 type=write size=2579947 2024-03-04T17:50:19.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.824+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=PAIINI8BS6AV type=del size=104744 2024-03-04T17:50:19.828 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.828+0000 7fe960f568c0 1 < completed write operation key=QGLN27WH1N2O tag=T1EQ49Z881WQ type=write size=73961 2024-03-04T17:50:19.835 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.832+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=8AG774GLISM3 type=write size=283392 2024-03-04T17:50:19.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.836+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=JAG4E91WB03T type=del 2024-03-04T17:50:19.843 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.840+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=NT3FJ8E237WM type=del size=1316508 2024-03-04T17:50:19.846 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.844+0000 7fe960f568c0 1 < canceled operation key=2FUTMG0SSXBY tag=SQMQTOJCGXAQ type=write size=1021641 2024-03-04T17:50:19.853 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.852+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~9K3TNH1D2EDP.0 size=299532 2024-03-04T17:50:19.853 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.852+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=9K3TNH1D2EDP tag=X2A1PBU4KBCH type=write size=1348108 2024-03-04T17:50:19.857 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.856+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=5MI39MDEVU09 type=del 2024-03-04T17:50:19.863 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.860+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=MJXOEE2AWYEZ type=del size=2472718 2024-03-04T17:50:19.863 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.860+0000 7fe960f568c0 1 < canceled part key=_multipart_FLYDZHOIWO6L.2~OMIKVR6R8WU1.0 size=1048576 2024-03-04T17:50:19.866 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.864+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~OMIKVR6R8WU1.1 size=590927 2024-03-04T17:50:19.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.872+0000 7fe960f568c0 1 < canceled multipart upload key=FLYDZHOIWO6L upload=OMIKVR6R8WU1 tag=81LKXF3RWC0Z type=write size=2688079 2024-03-04T17:50:19.877 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.876+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=J0VL7K4H1G19 type=del size=2639141 2024-03-04T17:50:19.880 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.880+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=O329XCYFBINL type=del 2024-03-04T17:50:19.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.888+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~N42HW107Y6AJ.0 size=708507 2024-03-04T17:50:19.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.888+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=N42HW107Y6AJ tag=EO9F0OVHY4Z2 type=write size=1757083 2024-03-04T17:50:19.892 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.892+0000 7fe960f568c0 1 < canceled operation key=NF4OIJWJ83DE tag=JSIC3IYC006R type=del size=2123651 2024-03-04T17:50:19.900 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.900+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~85KTK015YL1X.0 size=1048576 2024-03-04T17:50:19.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.900+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~85KTK015YL1X.1 size=1048576 2024-03-04T17:50:19.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.904+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~85KTK015YL1X.2 size=48300 2024-03-04T17:50:19.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.904+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=85KTK015YL1X tag=FJ371L72JAUR type=write size=3194028 2024-03-04T17:50:19.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.908+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=ZJU1Z10UIG3R type=del 2024-03-04T17:50:19.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.912+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=W8C2R1NFHRTM type=write size=66095 2024-03-04T17:50:19.917 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.916+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~JMC0DNP71VHQ.0 size=1048576 2024-03-04T17:50:19.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.920+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~JMC0DNP71VHQ.1 size=405123 2024-03-04T17:50:19.927 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.924+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=JMC0DNP71VHQ tag=GT9SDKU5JFNT type=write size=2502275 2024-03-04T17:50:19.933 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.932+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~FLGKHORVTRWV.0 size=1048576 2024-03-04T17:50:19.936 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.936+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~FLGKHORVTRWV.1 size=990442 2024-03-04T17:50:19.936 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.936+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=FLGKHORVTRWV tag=5D24ITLVON9C type=write size=3087594 2024-03-04T17:50:19.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.944+0000 7fe960f568c0 1 < suggested update operation key=QFF67JIW8Y8E tag=BH4DXN974PVT type=del size=2973098 2024-03-04T17:50:19.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.948+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=II0IVEUNA2GR type=del size=3327562 2024-03-04T17:50:19.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.952+0000 7fe960f568c0 1 < completed delete operation key=ASC735ZCBLCG tag=Z48JN4XYANMM type=del 2024-03-04T17:50:19.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.956+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=R7VY0IZJALFG type=del size=1659347 2024-03-04T17:50:19.965 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.964+0000 7fe960f568c0 1 < suggested update operation key=FLYDZHOIWO6L tag=VFE55VS8MNH9 type=write size=513713 2024-03-04T17:50:19.972 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.972+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~FCY7YXKYJ8VN.0 size=131482 2024-03-04T17:50:19.972 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.972+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=FCY7YXKYJ8VN tag=LB5WLPG511Z7 type=write size=1180058 2024-03-04T17:50:19.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.976+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~52HEK3BBPXH3.0 size=1048576 2024-03-04T17:50:19.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.980+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~52HEK3BBPXH3.1 size=434435 2024-03-04T17:50:19.987 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.984+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=52HEK3BBPXH3 tag=FYX4YMGBR6BK type=write size=2531587 2024-03-04T17:50:19.991 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.988+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=JEFRPDN5GVQQ type=del size=1343214 2024-03-04T17:50:19.994 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.992+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=3XBHC7PPHX27 type=del 2024-03-04T17:50:19.998 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.996+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=CEHJOA78V3QM type=del size=1865855 2024-03-04T17:50:19.999 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:19.996+0000 7fe960f568c0 1 < canceled part key=_multipart_JSO6WIYMPCF6.2~XF0HO9VUJ45N.0 size=1048576 2024-03-04T17:50:20.002 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.000+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~XF0HO9VUJ45N.1 size=1048576 2024-03-04T17:50:20.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.004+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~XF0HO9VUJ45N.2 size=790552 2024-03-04T17:50:20.013 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.012+0000 7fe960f568c0 1 < canceled multipart upload key=JSO6WIYMPCF6 upload=XF0HO9VUJ45N tag=IUYTMYSTWH1J type=write size=3936280 2024-03-04T17:50:20.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.016+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=WECH2N4YAJV2 type=del size=341310 2024-03-04T17:50:20.022 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.020+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=PAIINI8BS6AV type=del 2024-03-04T17:50:20.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.028+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~M3WL3QVUJ6P0.0 size=1048576 2024-03-04T17:50:20.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.032+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~M3WL3QVUJ6P0.1 size=186963 2024-03-04T17:50:20.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.032+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=M3WL3QVUJ6P0 tag=SDQOMKIEXQ1K type=write size=2284115 2024-03-04T17:50:20.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.036+0000 7fe960f568c0 1 < completed write operation key=2KZ4IFW54C5B tag=8AG774GLISM3 type=write size=283392 2024-03-04T17:50:20.044 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.044+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~1F5GWVFT3H6F.0 size=1048576 2024-03-04T17:50:20.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.044+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~1F5GWVFT3H6F.1 size=885447 2024-03-04T17:50:20.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.044+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=1F5GWVFT3H6F tag=P6LXGLF420YB type=write size=2982599 2024-03-04T17:50:20.055 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.056+0000 7fe960f568c0 1 < suggested remove operation key=7Z7BPZ8XGBHZ tag=NT3FJ8E237WM type=del 2024-03-04T17:50:20.060 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.060+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=NLWTV94EHHUX type=write size=864433 2024-03-04T17:50:20.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.064+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~9K3TNH1D2EDP.0 size=299532 2024-03-04T17:50:20.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.068+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=9K3TNH1D2EDP tag=X2A1PBU4KBCH type=write size=1348108 2024-03-04T17:50:20.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.072+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=G2VJHW19Q058 type=write size=557795 2024-03-04T17:50:20.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.080+0000 7fe960f568c0 1 < completed delete operation key=ASC735ZCBLCG tag=MJXOEE2AWYEZ type=del 2024-03-04T17:50:20.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.084+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=DFYF2VJ0ICHC type=del size=1834639 2024-03-04T17:50:20.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.092+0000 7fe960f568c0 1 < suggested remove operation key=8PZD01OOX3AF tag=J0VL7K4H1G19 type=del 2024-03-04T17:50:20.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.096+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=FVM3JKOXNXM1 type=del size=1939385 2024-03-04T17:50:20.101 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.100+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~N42HW107Y6AJ.0 size=708507 2024-03-04T17:50:20.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.104+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=N42HW107Y6AJ tag=EO9F0OVHY4Z2 type=write size=1757083 2024-03-04T17:50:20.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.112+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~N900329ITSFZ.0 size=1048576 2024-03-04T17:50:20.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.116+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~N900329ITSFZ.1 size=1048576 2024-03-04T17:50:20.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.120+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~N900329ITSFZ.2 size=578811 2024-03-04T17:50:20.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.120+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=N900329ITSFZ tag=2JYNZQHQD2WS type=write size=3724539 2024-03-04T17:50:20.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.120+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~85KTK015YL1X.0 size=1048576 2024-03-04T17:50:20.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.124+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~85KTK015YL1X.1 size=1048576 2024-03-04T17:50:20.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.128+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~85KTK015YL1X.2 size=48300 2024-03-04T17:50:20.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.136+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=85KTK015YL1X tag=FJ371L72JAUR type=write size=3194028 2024-03-04T17:50:20.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.140+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=Z88N1XWI1G1S type=del size=1437393 2024-03-04T17:50:20.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.144+0000 7fe960f568c0 1 < completed write operation key=KYSLL05AUHAC tag=W8C2R1NFHRTM type=write size=66095 2024-03-04T17:50:20.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.148+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=E5UP36P7YEA1 type=del size=1705730 2024-03-04T17:50:20.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.148+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~FLGKHORVTRWV.0 size=1048576 2024-03-04T17:50:20.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.152+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~FLGKHORVTRWV.1 size=990442 2024-03-04T17:50:20.159 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.160+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=FLGKHORVTRWV tag=5D24ITLVON9C type=write size=3087594 2024-03-04T17:50:20.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.164+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=8J3N2K6UOOOP type=del size=1772124 2024-03-04T17:50:20.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.172+0000 7fe960f568c0 1 < suggested remove operation key=7Z7BPZ8XGBHZ tag=II0IVEUNA2GR type=del 2024-03-04T17:50:20.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.176+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=OMK8MF1D6N80 type=del size=3901157 2024-03-04T17:50:20.180 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.180+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=R7VY0IZJALFG type=del 2024-03-04T17:50:20.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.188+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~BD79VP61R4TF.0 size=1048576 2024-03-04T17:50:20.190 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.188+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~BD79VP61R4TF.1 size=349216 2024-03-04T17:50:20.190 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.188+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=BD79VP61R4TF tag=YG6V24YMWII3 type=write size=2446368 2024-03-04T17:50:20.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.192+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~FCY7YXKYJ8VN.0 size=131482 2024-03-04T17:50:20.201 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.200+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=FCY7YXKYJ8VN tag=LB5WLPG511Z7 type=write size=1180058 2024-03-04T17:50:20.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.212+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~18VIJ3ISZ1XI.0 size=1048576 2024-03-04T17:50:20.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.212+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~18VIJ3ISZ1XI.1 size=747561 2024-03-04T17:50:20.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.212+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=18VIJ3ISZ1XI tag=EO8R3EA6XSD7 type=write size=2844713 2024-03-04T17:50:20.218 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.216+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=JEFRPDN5GVQQ type=del 2024-03-04T17:50:20.222 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.220+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=YHCZI3ZMG1AE type=del size=2689301 2024-03-04T17:50:20.230 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.228+0000 7fe960f568c0 1 < suggested remove operation key=2MKQA5JYWU8L tag=CEHJOA78V3QM type=del 2024-03-04T17:50:20.234 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.232+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=V27R9W7ERFQ8 type=del size=2799391 2024-03-04T17:50:20.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.236+0000 7fe960f568c0 -1 < failed to complete operation key=7Z7BPZ8XGBHZ tag=WECH2N4YAJV2 type=del size=341310: (22) Invalid argument 2024-03-04T17:50:20.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.244+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~LV0IQ7U6TNQT.0 size=1048576 2024-03-04T17:50:20.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.248+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~LV0IQ7U6TNQT.1 size=683006 2024-03-04T17:50:20.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.248+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=LV0IQ7U6TNQT tag=U902CQNVP5QJ type=write size=2780158 2024-03-04T17:50:20.252 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.252+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~M3WL3QVUJ6P0.0 size=1048576 2024-03-04T17:50:20.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.252+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~M3WL3QVUJ6P0.1 size=186963 2024-03-04T17:50:20.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.260+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=M3WL3QVUJ6P0 tag=SDQOMKIEXQ1K type=write size=2284115 2024-03-04T17:50:20.271 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.268+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~5OQYQVWGMJOJ.0 size=1040822 2024-03-04T17:50:20.271 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.268+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=5OQYQVWGMJOJ tag=M8UBCGC3UQYQ type=write size=2089398 2024-03-04T17:50:20.271 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.268+0000 7fe960f568c0 1 < canceled part key=_multipart_BFF9OD00TU54.2~1F5GWVFT3H6F.0 size=1048576 2024-03-04T17:50:20.273 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.272+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~1F5GWVFT3H6F.1 size=885447 2024-03-04T17:50:20.279 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.280+0000 7fe960f568c0 1 < canceled multipart upload key=BFF9OD00TU54 upload=1F5GWVFT3H6F tag=P6LXGLF420YB type=write size=2982599 2024-03-04T17:50:20.283 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.284+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=EQICXXXLBMFS type=del size=2573462 2024-03-04T17:50:20.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.288+0000 7fe960f568c0 1 < completed write operation key=JP3FRB6MD7B4 tag=NLWTV94EHHUX type=write size=864433 2024-03-04T17:50:20.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.292+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~VLIJNFONR9DM.0 size=1048576 2024-03-04T17:50:20.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.296+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~VLIJNFONR9DM.1 size=1048576 2024-03-04T17:50:20.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.296+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~VLIJNFONR9DM.2 size=962389 2024-03-04T17:50:20.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.296+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=VLIJNFONR9DM tag=PZTZ6NOP1U5V type=write size=4108117 2024-03-04T17:50:20.304 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.304+0000 7fe960f568c0 1 < completed write operation key=MIZRUP4MEA0J tag=G2VJHW19Q058 type=write size=557795 2024-03-04T17:50:20.307 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.308+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=S7C1L822L433 type=del size=424816 2024-03-04T17:50:20.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.312+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=DFYF2VJ0ICHC type=del 2024-03-04T17:50:20.317 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.316+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=T0CJEOSGCDQW type=write size=481626 2024-03-04T17:50:20.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.324+0000 7fe960f568c0 1 < suggested update operation key=3X4719GHK3BI tag=FVM3JKOXNXM1 type=del size=1939385 2024-03-04T17:50:20.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.332+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~BE6TGX3TM3CU.0 size=1048576 2024-03-04T17:50:20.334 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.332+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~BE6TGX3TM3CU.1 size=996695 2024-03-04T17:50:20.334 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.332+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=BE6TGX3TM3CU tag=WMI9WT3MNPOM type=write size=3093847 2024-03-04T17:50:20.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.336+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~N900329ITSFZ.0 size=1048576 2024-03-04T17:50:20.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.340+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~N900329ITSFZ.1 size=1048576 2024-03-04T17:50:20.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.344+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~N900329ITSFZ.2 size=578811 2024-03-04T17:50:20.351 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.352+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=N900329ITSFZ tag=2JYNZQHQD2WS type=write size=3724539 2024-03-04T17:50:20.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.356+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=SY5093Y6CQUW type=write size=150782 2024-03-04T17:50:20.360 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.360+0000 7fe960f568c0 1 < canceled operation key=JP3FRB6MD7B4 tag=Z88N1XWI1G1S type=del size=1437393 2024-03-04T17:50:20.365 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.364+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=SUI6QLZQ4O24 type=del size=3290902 2024-03-04T17:50:20.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.372+0000 7fe960f568c0 1 < suggested remove operation key=7Z7BPZ8XGBHZ tag=E5UP36P7YEA1 type=del 2024-03-04T17:50:20.375 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.376+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=VE4KCPOM3KUB type=del size=402197 2024-03-04T17:50:20.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.380+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=8J3N2K6UOOOP type=del 2024-03-04T17:50:20.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.384+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~P0LHXJ61XAI1.0 size=410034 2024-03-04T17:50:20.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.384+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=P0LHXJ61XAI1 tag=97UFYBCK409L type=write size=1458610 2024-03-04T17:50:20.390 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.388+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=OMK8MF1D6N80 type=del 2024-03-04T17:50:20.393 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.392+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=MQVMTS1DA5UV type=del size=632346 2024-03-04T17:50:20.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.396+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~BD79VP61R4TF.0 size=1048576 2024-03-04T17:50:20.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.396+0000 7fe960f568c0 1 < canceled part key=_multipart_NF4OIJWJ83DE.2~BD79VP61R4TF.1 size=349216 2024-03-04T17:50:20.404 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.404+0000 7fe960f568c0 1 < canceled multipart upload key=NF4OIJWJ83DE upload=BD79VP61R4TF tag=YG6V24YMWII3 type=write size=2446368 2024-03-04T17:50:20.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.408+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=N0VP64BUPSQT type=del size=4064443 2024-03-04T17:50:20.412 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.412+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~18VIJ3ISZ1XI.0 size=1048576 2024-03-04T17:50:20.415 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.412+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~18VIJ3ISZ1XI.1 size=747561 2024-03-04T17:50:20.421 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.420+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=18VIJ3ISZ1XI tag=EO8R3EA6XSD7 type=write size=2844713 2024-03-04T17:50:20.424 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.424+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=3FDSMCPMTRLY type=write size=348696 2024-03-04T17:50:20.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.428+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=YHCZI3ZMG1AE type=del 2024-03-04T17:50:20.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.432+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=R66IM48MQQML type=del size=3154833 2024-03-04T17:50:20.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.436+0000 7fe960f568c0 -1 < failed to complete operation key=7Z7BPZ8XGBHZ tag=V27R9W7ERFQ8 type=del size=2799391: (22) Invalid argument 2024-03-04T17:50:20.442 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.440+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=XXOG101HS64H type=del size=3438017 2024-03-04T17:50:20.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.444+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~LV0IQ7U6TNQT.0 size=1048576 2024-03-04T17:50:20.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.448+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~LV0IQ7U6TNQT.1 size=683006 2024-03-04T17:50:20.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.456+0000 7fe960f568c0 1 < completed multipart upload key=2MKQA5JYWU8L upload=LV0IQ7U6TNQT tag=U902CQNVP5QJ type=write size=2780158 2024-03-04T17:50:20.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.464+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~YWI3GN7VRQI9.0 size=875958 2024-03-04T17:50:20.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.464+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=YWI3GN7VRQI9 tag=HL0ROTAIEHSM type=write size=1924534 2024-03-04T17:50:20.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.468+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~5OQYQVWGMJOJ.0 size=1040822 2024-03-04T17:50:20.477 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.476+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=5OQYQVWGMJOJ tag=M8UBCGC3UQYQ type=write size=2089398 2024-03-04T17:50:20.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.484+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~RNQC6SRD2W1C.0 size=1048576 2024-03-04T17:50:20.487 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.488+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~RNQC6SRD2W1C.1 size=359294 2024-03-04T17:50:20.487 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.488+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=RNQC6SRD2W1C tag=NAG5K71ZECVP type=write size=2456446 2024-03-04T17:50:20.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.492+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=EQICXXXLBMFS type=del 2024-03-04T17:50:20.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.492+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=2BITQNE1OPHJ type=del size=2208481 2024-03-04T17:50:20.499 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.496+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~VLIJNFONR9DM.0 size=1048576 2024-03-04T17:50:20.502 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.500+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~VLIJNFONR9DM.1 size=1048576 2024-03-04T17:50:20.502 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.500+0000 7fe960f568c0 1 < canceled part key=_multipart_FLYDZHOIWO6L.2~VLIJNFONR9DM.2 size=962389 2024-03-04T17:50:20.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.508+0000 7fe960f568c0 1 < canceled multipart upload key=FLYDZHOIWO6L upload=VLIJNFONR9DM tag=PZTZ6NOP1U5V type=write size=4108117 2024-03-04T17:50:20.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.512+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=KAT5V5147PXL type=write size=219958 2024-03-04T17:50:20.515 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.516+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=S7C1L822L433 type=del 2024-03-04T17:50:20.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.516+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=VT4Y5QHX4HVT type=del size=992146 2024-03-04T17:50:20.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.520+0000 7fe960f568c0 1 < completed write operation key=GZ4RRIR5CC26 tag=T0CJEOSGCDQW type=write size=481626 2024-03-04T17:50:20.529 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.528+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~OVMJ25Q88HDE.0 size=1048576 2024-03-04T17:50:20.532 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.532+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~OVMJ25Q88HDE.1 size=1048576 2024-03-04T17:50:20.535 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.536+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~OVMJ25Q88HDE.2 size=717490 2024-03-04T17:50:20.536 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.536+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=OVMJ25Q88HDE tag=SG6YUCLCXFRS type=write size=3863218 2024-03-04T17:50:20.536 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.536+0000 7fe960f568c0 1 < canceled part key=_multipart_6X4ZV3BFZNDL.2~BE6TGX3TM3CU.0 size=1048576 2024-03-04T17:50:20.539 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.536+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~BE6TGX3TM3CU.1 size=996695 2024-03-04T17:50:20.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.544+0000 7fe960f568c0 1 < canceled multipart upload key=6X4ZV3BFZNDL upload=BE6TGX3TM3CU tag=WMI9WT3MNPOM type=write size=3093847 2024-03-04T17:50:20.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.552+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~JI4111XHRACN.0 size=517634 2024-03-04T17:50:20.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.552+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=JI4111XHRACN tag=8L5AAZDHJSPN type=write size=1566210 2024-03-04T17:50:20.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.556+0000 7fe960f568c0 1 < completed write operation key=78N8HHC4O4CG tag=SY5093Y6CQUW type=write size=150782 2024-03-04T17:50:20.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.560+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~SK7QJIHK2USG.0 size=1048576 2024-03-04T17:50:20.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.564+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~SK7QJIHK2USG.1 size=1048576 2024-03-04T17:50:20.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.572+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~SK7QJIHK2USG.2 size=654903 2024-03-04T17:50:20.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.572+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=SK7QJIHK2USG tag=B1AT86DY1XTY type=write size=3800631 2024-03-04T17:50:20.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.576+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=SUI6QLZQ4O24 type=del 2024-03-04T17:50:20.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.580+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=ZYW2U4PSBWSU type=write size=195764 2024-03-04T17:50:20.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.584+0000 7fe960f568c0 1 < completed delete operation key=U2G6YEVD5506 tag=VE4KCPOM3KUB type=del 2024-03-04T17:50:20.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.584+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=3TAUNWJE1IOO type=del size=2307757 2024-03-04T17:50:20.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.588+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~P0LHXJ61XAI1.0 size=410034 2024-03-04T17:50:20.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.596+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=P0LHXJ61XAI1 tag=97UFYBCK409L type=write size=1458610 2024-03-04T17:50:20.604 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.604+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~TEEHSAVOUWXB.0 size=1048576 2024-03-04T17:50:20.607 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.604+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~TEEHSAVOUWXB.1 size=487743 2024-03-04T17:50:20.607 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.604+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=TEEHSAVOUWXB tag=KOU14582E74R type=write size=2584895 2024-03-04T17:50:20.611 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.608+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=MQVMTS1DA5UV type=del 2024-03-04T17:50:20.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.612+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=7WP2JATCN0X5 type=del size=4153532 2024-03-04T17:50:20.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.620+0000 7fe960f568c0 1 < suggested update operation key=7Z7BPZ8XGBHZ tag=N0VP64BUPSQT type=del size=4064443 2024-03-04T17:50:20.626 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.624+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=UVV2BYN46M9L type=del size=54717 2024-03-04T17:50:20.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.628+0000 7fe960f568c0 1 < completed write operation key=78N8HHC4O4CG tag=3FDSMCPMTRLY type=write size=348696 2024-03-04T17:50:20.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.632+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=T08D1B40KFCE type=del size=4190059 2024-03-04T17:50:20.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.640+0000 7fe960f568c0 1 < suggested update operation key=UEVMIPQ9FR81 tag=R66IM48MQQML type=del size=3154833 2024-03-04T17:50:20.646 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.644+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=5WUF9B29PXO1 type=del size=1947731 2024-03-04T17:50:20.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.648+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=XXOG101HS64H type=del 2024-03-04T17:50:20.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.656+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~4K7FFDZKKJ6Q.0 size=178927 2024-03-04T17:50:20.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.656+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=4K7FFDZKKJ6Q tag=1PNYBW5BBDYA type=write size=1227503 2024-03-04T17:50:20.661 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.660+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~YWI3GN7VRQI9.0 size=875958 2024-03-04T17:50:20.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.664+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=YWI3GN7VRQI9 tag=HL0ROTAIEHSM type=write size=1924534 2024-03-04T17:50:20.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.672+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~V1E369V3615U.0 size=967628 2024-03-04T17:50:20.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.672+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=V1E369V3615U tag=7JYJ2WA32AEO type=write size=2016204 2024-03-04T17:50:20.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.676+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~RNQC6SRD2W1C.0 size=1048576 2024-03-04T17:50:20.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.680+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~RNQC6SRD2W1C.1 size=359294 2024-03-04T17:50:20.686 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.684+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=RNQC6SRD2W1C tag=NAG5K71ZECVP type=write size=2456446 2024-03-04T17:50:20.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.688+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=0IQA33CY8DRF type=del size=4087351 2024-03-04T17:50:20.693 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.692+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=2BITQNE1OPHJ type=del 2024-03-04T17:50:20.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.696+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=D3L0ZY1RE7J4 type=del size=443218 2024-03-04T17:50:20.701 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.700+0000 7fe960f568c0 -1 < failed to complete operation key=UEVMIPQ9FR81 tag=KAT5V5147PXL type=write size=219958: (22) Invalid argument 2024-03-04T17:50:20.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.704+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=C8ZCD749HEM0 type=del size=275346 2024-03-04T17:50:20.712 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.712+0000 7fe960f568c0 1 < suggested update operation key=NF4OIJWJ83DE tag=VT4Y5QHX4HVT type=del size=992146 2024-03-04T17:50:20.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.716+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=LT5JQSUYXPAF type=del size=2575547 2024-03-04T17:50:20.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.720+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~OVMJ25Q88HDE.0 size=1048576 2024-03-04T17:50:20.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.724+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~OVMJ25Q88HDE.1 size=1048576 2024-03-04T17:50:20.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.728+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~OVMJ25Q88HDE.2 size=717490 2024-03-04T17:50:20.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.732+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=OVMJ25Q88HDE tag=SG6YUCLCXFRS type=write size=3863218 2024-03-04T17:50:20.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.740+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~NKQ1B2HBIXWE.0 size=1048576 2024-03-04T17:50:20.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.744+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~NKQ1B2HBIXWE.1 size=1048576 2024-03-04T17:50:20.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.744+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~NKQ1B2HBIXWE.2 size=319300 2024-03-04T17:50:20.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.744+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=NKQ1B2HBIXWE tag=RBFYDGTA0NED type=write size=3465028 2024-03-04T17:50:20.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.748+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~JI4111XHRACN.0 size=517634 2024-03-04T17:50:20.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.756+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=JI4111XHRACN tag=8L5AAZDHJSPN type=write size=1566210 2024-03-04T17:50:20.760 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.764+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=R369B45OCO1Q type=write size=448350 2024-03-04T17:50:20.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.764+0000 7fe960f568c0 1 < canceled part key=_multipart_QFF67JIW8Y8E.2~SK7QJIHK2USG.0 size=1048576 2024-03-04T17:50:20.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.764+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~SK7QJIHK2USG.1 size=1048576 2024-03-04T17:50:20.767 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.764+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~SK7QJIHK2USG.2 size=654903 2024-03-04T17:50:20.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.772+0000 7fe960f568c0 1 < canceled multipart upload key=QFF67JIW8Y8E upload=SK7QJIHK2USG tag=B1AT86DY1XTY type=write size=3800631 2024-03-04T17:50:20.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.776+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=JREMHZRYS348 type=write size=522603 2024-03-04T17:50:20.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.780+0000 7fe960f568c0 1 < canceled operation key=768X2XKWJW1S tag=ZYW2U4PSBWSU type=write size=195764 2024-03-04T17:50:20.789 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.788+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~SV0Q3LLYUHDW.0 size=1048576 2024-03-04T17:50:20.791 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.792+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~SV0Q3LLYUHDW.1 size=1048576 2024-03-04T17:50:20.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.792+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~SV0Q3LLYUHDW.2 size=80564 2024-03-04T17:50:20.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.792+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=SV0Q3LLYUHDW tag=05Y9YZ50J0BL type=write size=3226292 2024-03-04T17:50:20.798 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.796+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=3TAUNWJE1IOO type=del 2024-03-04T17:50:20.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.804+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~8V935LE4H5K7.0 size=1048576 2024-03-04T17:50:20.808 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.808+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~8V935LE4H5K7.1 size=1048576 2024-03-04T17:50:20.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.808+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~8V935LE4H5K7.2 size=276980 2024-03-04T17:50:20.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.808+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=8V935LE4H5K7 tag=1SMTKKVNH2AB type=write size=3422708 2024-03-04T17:50:20.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.812+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~TEEHSAVOUWXB.0 size=1048576 2024-03-04T17:50:20.818 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.816+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~TEEHSAVOUWXB.1 size=487743 2024-03-04T17:50:20.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.824+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=TEEHSAVOUWXB tag=KOU14582E74R type=write size=2584895 2024-03-04T17:50:20.831 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.832+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~3KH3RJASUBTV.0 size=1048576 2024-03-04T17:50:20.834 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.832+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~3KH3RJASUBTV.1 size=1048576 2024-03-04T17:50:20.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.836+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~3KH3RJASUBTV.2 size=538857 2024-03-04T17:50:20.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.836+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=3KH3RJASUBTV tag=XBFGCZEI0337 type=write size=3684585 2024-03-04T17:50:20.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.840+0000 7fe960f568c0 -1 < failed to complete operation key=UEVMIPQ9FR81 tag=7WP2JATCN0X5 type=del size=4153532: (22) Invalid argument 2024-03-04T17:50:20.845 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.844+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=OF11R3AMA4A5 type=del size=1230856 2024-03-04T17:50:20.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.848+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=UVV2BYN46M9L type=del 2024-03-04T17:50:20.853 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.852+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=Z0KHP66OUJW0 type=del size=217017 2024-03-04T17:50:20.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.856+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=T08D1B40KFCE type=del 2024-03-04T17:50:20.861 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.860+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=MR8HN1BTQYPE type=del size=462886 2024-03-04T17:50:20.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.868+0000 7fe960f568c0 1 < suggested remove operation key=GZ4RRIR5CC26 tag=5WUF9B29PXO1 type=del 2024-03-04T17:50:20.876 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.876+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~ILHQVUZN5UKE.0 size=1048576 2024-03-04T17:50:20.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.880+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~ILHQVUZN5UKE.1 size=1048576 2024-03-04T17:50:20.884 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.880+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~ILHQVUZN5UKE.2 size=380445 2024-03-04T17:50:20.884 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.880+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=ILHQVUZN5UKE tag=39QYP1SUUJ8T type=write size=3526173 2024-03-04T17:50:20.887 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.884+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~4K7FFDZKKJ6Q.0 size=178927 2024-03-04T17:50:20.892 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.892+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=4K7FFDZKKJ6Q tag=1PNYBW5BBDYA type=write size=1227503 2024-03-04T17:50:20.896 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.896+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=6AYX1ZAM7CHJ type=del size=2295109 2024-03-04T17:50:20.899 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.900+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~V1E369V3615U.0 size=967628 2024-03-04T17:50:20.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.904+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=V1E369V3615U tag=7JYJ2WA32AEO type=write size=2016204 2024-03-04T17:50:20.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.908+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=2HOKY4GUT2ZV type=del size=1387112 2024-03-04T17:50:20.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.912+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=0IQA33CY8DRF type=del 2024-03-04T17:50:20.916 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.916+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=05FEVBQM4ED5 type=write size=790434 2024-03-04T17:50:20.926 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.924+0000 7fe960f568c0 1 < suggested update operation key=NF4OIJWJ83DE tag=D3L0ZY1RE7J4 type=del size=443218 2024-03-04T17:50:20.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.928+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=F2BSDUYZHXJA type=del size=3858724 2024-03-04T17:50:20.934 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.932+0000 7fe960f568c0 1 < canceled operation key=8PZD01OOX3AF tag=C8ZCD749HEM0 type=del size=275346 2024-03-04T17:50:20.937 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.936+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=91ZXQ0JEWU14 type=write size=332612 2024-03-04T17:50:20.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.944+0000 7fe960f568c0 1 < suggested remove operation key=R6ZDVOUE9NXJ tag=LT5JQSUYXPAF type=del 2024-03-04T17:50:20.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.948+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=EIL8MH1FYSTP type=del size=1777105 2024-03-04T17:50:20.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.952+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~NKQ1B2HBIXWE.0 size=1048576 2024-03-04T17:50:20.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.952+0000 7fe960f568c0 1 < canceled part key=_multipart_FLYDZHOIWO6L.2~NKQ1B2HBIXWE.1 size=1048576 2024-03-04T17:50:20.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.956+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~NKQ1B2HBIXWE.2 size=319300 2024-03-04T17:50:20.963 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.960+0000 7fe960f568c0 1 < canceled multipart upload key=FLYDZHOIWO6L upload=NKQ1B2HBIXWE tag=RBFYDGTA0NED type=write size=3465028 2024-03-04T17:50:20.966 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.964+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=7B4D9EWF9W0T type=del size=3043773 2024-03-04T17:50:20.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.968+0000 7fe960f568c0 1 < completed write operation key=XSQ3GE2UHMZG tag=R369B45OCO1Q type=write size=448350 2024-03-04T17:50:20.975 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.972+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=IHLBGBKE0RZV type=write size=752084 2024-03-04T17:50:20.978 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.976+0000 7fe960f568c0 1 < completed write operation key=FSWBX523X9V5 tag=JREMHZRYS348 type=write size=522603 2024-03-04T17:50:20.983 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.980+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=1JXADFRXW2IV type=write size=815248 2024-03-04T17:50:20.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.984+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~SV0Q3LLYUHDW.0 size=1048576 2024-03-04T17:50:20.989 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.988+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~SV0Q3LLYUHDW.1 size=1048576 2024-03-04T17:50:20.992 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.992+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~SV0Q3LLYUHDW.2 size=80564 2024-03-04T17:50:20.998 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:20.996+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=SV0Q3LLYUHDW tag=05Y9YZ50J0BL type=write size=3226292 2024-03-04T17:50:21.002 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.000+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=E8C1RILUF4VO type=del size=692877 2024-03-04T17:50:21.003 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.000+0000 7fe960f568c0 1 < canceled part key=_multipart_JJ7SM1Y73A4C.2~8V935LE4H5K7.0 size=1048576 2024-03-04T17:50:21.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.004+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~8V935LE4H5K7.1 size=1048576 2024-03-04T17:50:21.009 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.008+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~8V935LE4H5K7.2 size=276980 2024-03-04T17:50:21.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.012+0000 7fe960f568c0 1 < canceled multipart upload key=JJ7SM1Y73A4C upload=8V935LE4H5K7 tag=1SMTKKVNH2AB type=write size=3422708 2024-03-04T17:50:21.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.020+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~4F3H7ZSBRX9S.0 size=1048576 2024-03-04T17:50:21.023 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.024+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~4F3H7ZSBRX9S.1 size=1048576 2024-03-04T17:50:21.026 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.024+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~4F3H7ZSBRX9S.2 size=287195 2024-03-04T17:50:21.026 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.024+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=4F3H7ZSBRX9S tag=HZQUPQLCHCUD type=write size=3432923 2024-03-04T17:50:21.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.028+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~3KH3RJASUBTV.0 size=1048576 2024-03-04T17:50:21.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.032+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~3KH3RJASUBTV.1 size=1048576 2024-03-04T17:50:21.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.036+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~3KH3RJASUBTV.2 size=538857 2024-03-04T17:50:21.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.040+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=3KH3RJASUBTV tag=XBFGCZEI0337 type=write size=3684585 2024-03-04T17:50:21.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.044+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=51W111VB6QGR type=write size=1042831 2024-03-04T17:50:21.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.048+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=OF11R3AMA4A5 type=del 2024-03-04T17:50:21.054 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.052+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=QLRAF6ZHD952 type=del size=2486978 2024-03-04T17:50:21.060 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.060+0000 7fe960f568c0 1 < suggested update operation key=U2G6YEVD5506 tag=Z0KHP66OUJW0 type=del size=217017 2024-03-04T17:50:21.067 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.068+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~HJWICBDCD7MG.0 size=734126 2024-03-04T17:50:21.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.068+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=HJWICBDCD7MG tag=L8I7HQYFQVDY type=write size=1782702 2024-03-04T17:50:21.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.068+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=MR8HN1BTQYPE type=del 2024-03-04T17:50:21.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.072+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=3FY7WOLPTQID type=del size=1758401 2024-03-04T17:50:21.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.076+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~ILHQVUZN5UKE.0 size=1048576 2024-03-04T17:50:21.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.076+0000 7fe960f568c0 1 < canceled part key=_multipart_FLYDZHOIWO6L.2~ILHQVUZN5UKE.1 size=1048576 2024-03-04T17:50:21.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.080+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~ILHQVUZN5UKE.2 size=380445 2024-03-04T17:50:21.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.088+0000 7fe960f568c0 1 < canceled multipart upload key=FLYDZHOIWO6L upload=ILHQVUZN5UKE tag=39QYP1SUUJ8T type=write size=3526173 2024-03-04T17:50:21.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.092+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=K60W9M6J4IHC type=write size=310061 2024-03-04T17:50:21.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.096+0000 7fe960f568c0 1 < canceled operation key=JP3FRB6MD7B4 tag=6AYX1ZAM7CHJ type=del size=2295109 2024-03-04T17:50:21.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.100+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=JOOG7P5I1QSS type=del size=4055905 2024-03-04T17:50:21.103 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.100+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=2HOKY4GUT2ZV type=del 2024-03-04T17:50:21.109 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.108+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~PMORW2CQ7RXX.0 size=1048576 2024-03-04T17:50:21.112 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.112+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~PMORW2CQ7RXX.1 size=120425 2024-03-04T17:50:21.112 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.112+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=PMORW2CQ7RXX tag=WBB1U5TIB281 type=write size=2217577 2024-03-04T17:50:21.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.116+0000 7fe960f568c0 1 < completed write operation key=QFF67JIW8Y8E tag=05FEVBQM4ED5 type=write size=790434 2024-03-04T17:50:21.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.116+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=BS44AZE84UF1 type=del size=547494 2024-03-04T17:50:21.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.120+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=F2BSDUYZHXJA type=del 2024-03-04T17:50:21.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.128+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~0MQ08JOC81D0.0 size=1048576 2024-03-04T17:50:21.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.132+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~0MQ08JOC81D0.1 size=361691 2024-03-04T17:50:21.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.132+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=0MQ08JOC81D0 tag=BPB9OTNTG920 type=write size=2458843 2024-03-04T17:50:21.136 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.136+0000 7fe960f568c0 1 < completed write operation key=KYSLL05AUHAC tag=91ZXQ0JEWU14 type=write size=332612 2024-03-04T17:50:21.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.144+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~R7Z9L12A0ZXD.0 size=616621 2024-03-04T17:50:21.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.144+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=R7Z9L12A0ZXD tag=XAD1I86FF82F type=write size=1665197 2024-03-04T17:50:21.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.148+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=EIL8MH1FYSTP type=del 2024-03-04T17:50:21.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.152+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~EMVGYLNMI9VP.0 size=1048576 2024-03-04T17:50:21.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.156+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~EMVGYLNMI9VP.1 size=461137 2024-03-04T17:50:21.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.156+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=EMVGYLNMI9VP tag=A8WS7UESEGW0 type=write size=2558289 2024-03-04T17:50:21.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.164+0000 7fe960f568c0 1 < suggested remove operation key=3X4719GHK3BI tag=7B4D9EWF9W0T type=del 2024-03-04T17:50:21.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.172+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~QOID61CXZ2SQ.0 size=1048576 2024-03-04T17:50:21.174 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.172+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~QOID61CXZ2SQ.1 size=1048576 2024-03-04T17:50:21.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.176+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~QOID61CXZ2SQ.2 size=196031 2024-03-04T17:50:21.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.176+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=QOID61CXZ2SQ tag=HSK48GHXP3IM type=write size=3341759 2024-03-04T17:50:21.181 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.180+0000 7fe960f568c0 -1 < failed to complete operation key=U2G6YEVD5506 tag=IHLBGBKE0RZV type=write size=752084: (22) Invalid argument 2024-03-04T17:50:21.185 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.184+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=GW74Y83B0ZHS type=del size=2569776 2024-03-04T17:50:21.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.192+0000 7fe960f568c0 1 < suggested remove operation key=FSWBX523X9V5 tag=1JXADFRXW2IV type=write 2024-03-04T17:50:21.196 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.196+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=8VJXZPXQCAKI type=del size=356282 2024-03-04T17:50:21.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.200+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=E8C1RILUF4VO type=del 2024-03-04T17:50:21.204 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.204+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=L2GSHT73OU80 type=del size=357252 2024-03-04T17:50:21.208 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.208+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~4F3H7ZSBRX9S.0 size=1048576 2024-03-04T17:50:21.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.208+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~4F3H7ZSBRX9S.1 size=1048576 2024-03-04T17:50:21.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.212+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~4F3H7ZSBRX9S.2 size=287195 2024-03-04T17:50:21.220 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.220+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=4F3H7ZSBRX9S tag=HZQUPQLCHCUD type=write size=3432923 2024-03-04T17:50:21.227 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.224+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~CKZVE1K5KK4O.0 size=1048576 2024-03-04T17:50:21.230 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.228+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~CKZVE1K5KK4O.1 size=366307 2024-03-04T17:50:21.230 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.228+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=CKZVE1K5KK4O tag=7ZWRWGDACQ7I type=write size=2463459 2024-03-04T17:50:21.234 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.232+0000 7fe960f568c0 1 < canceled operation key=JSO6WIYMPCF6 tag=51W111VB6QGR type=write size=1042831 2024-03-04T17:50:21.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.236+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=0YPNV4OMWUX7 type=del size=487212 2024-03-04T17:50:21.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.240+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=QLRAF6ZHD952 type=del 2024-03-04T17:50:21.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.248+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~KZPT64LNLQBT.0 size=518261 2024-03-04T17:50:21.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.248+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=KZPT64LNLQBT tag=6YMUSL589YVL type=write size=1566837 2024-03-04T17:50:21.252 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.252+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~HJWICBDCD7MG.0 size=734126 2024-03-04T17:50:21.257 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.256+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=HJWICBDCD7MG tag=L8I7HQYFQVDY type=write size=1782702 2024-03-04T17:50:21.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.264+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~6Z93VLJ2Y96P.0 size=350632 2024-03-04T17:50:21.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.264+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=6Z93VLJ2Y96P tag=DN5L09S9C6I7 type=write size=1399208 2024-03-04T17:50:21.268 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.268+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=3FY7WOLPTQID type=del 2024-03-04T17:50:21.277 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.276+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~JGUJDMXSNWX9.0 size=1048576 2024-03-04T17:50:21.280 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.280+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~JGUJDMXSNWX9.1 size=1048576 2024-03-04T17:50:21.283 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.280+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~JGUJDMXSNWX9.2 size=297692 2024-03-04T17:50:21.283 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.280+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=JGUJDMXSNWX9 tag=1NVWLC6ASLSX type=write size=3443420 2024-03-04T17:50:21.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.288+0000 7fe960f568c0 1 < suggested update operation key=QGLN27WH1N2O tag=K60W9M6J4IHC type=write size=310061 2024-03-04T17:50:21.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.292+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=D40KBKV2QG5T type=del size=1288937 2024-03-04T17:50:21.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.296+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=JOOG7P5I1QSS type=del 2024-03-04T17:50:21.302 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.300+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=ZGFW1TA9880D type=del size=2856070 2024-03-04T17:50:21.306 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.304+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~PMORW2CQ7RXX.0 size=1048576 2024-03-04T17:50:21.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.308+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~PMORW2CQ7RXX.1 size=120425 2024-03-04T17:50:21.315 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.312+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=PMORW2CQ7RXX tag=WBB1U5TIB281 type=write size=2217577 2024-03-04T17:50:21.318 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.316+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=RNTZ1ZP427F4 type=del size=212173 2024-03-04T17:50:21.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.320+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=BS44AZE84UF1 type=del 2024-03-04T17:50:21.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.328+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~7KE5EL45N9LT.0 size=289280 2024-03-04T17:50:21.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.328+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=7KE5EL45N9LT tag=WZQ3I03C2D5B type=write size=1337856 2024-03-04T17:50:21.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.332+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~0MQ08JOC81D0.0 size=1048576 2024-03-04T17:50:21.336 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.336+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~0MQ08JOC81D0.1 size=361691 2024-03-04T17:50:21.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.340+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=0MQ08JOC81D0 tag=BPB9OTNTG920 type=write size=2458843 2024-03-04T17:50:21.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.348+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~6D0V064QR0B2.0 size=1048576 2024-03-04T17:50:21.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.352+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~6D0V064QR0B2.1 size=1048576 2024-03-04T17:50:21.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.352+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~6D0V064QR0B2.2 size=750140 2024-03-04T17:50:21.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.352+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=6D0V064QR0B2 tag=PCSM3VBKL9UQ type=write size=3895868 2024-03-04T17:50:21.358 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.356+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~R7Z9L12A0ZXD.0 size=616621 2024-03-04T17:50:21.364 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.364+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=R7Z9L12A0ZXD tag=XAD1I86FF82F type=write size=1665197 2024-03-04T17:50:21.368 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.368+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=Z4QOTJJ2GLCS type=del size=2991514 2024-03-04T17:50:21.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.372+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~EMVGYLNMI9VP.0 size=1048576 2024-03-04T17:50:21.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.372+0000 7fe960f568c0 1 < canceled part key=_multipart_BFF9OD00TU54.2~EMVGYLNMI9VP.1 size=461137 2024-03-04T17:50:21.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.376+0000 7fe960f568c0 1 < canceled multipart upload key=BFF9OD00TU54 upload=EMVGYLNMI9VP tag=A8WS7UESEGW0 type=write size=2558289 2024-03-04T17:50:21.384 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.384+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~WG8CM8YT13S0.0 size=1048576 2024-03-04T17:50:21.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.388+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~WG8CM8YT13S0.1 size=922461 2024-03-04T17:50:21.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.388+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=WG8CM8YT13S0 tag=30S6SCX7KQEJ type=write size=3019613 2024-03-04T17:50:21.392 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.392+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~QOID61CXZ2SQ.0 size=1048576 2024-03-04T17:50:21.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.392+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~QOID61CXZ2SQ.1 size=1048576 2024-03-04T17:50:21.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.396+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~QOID61CXZ2SQ.2 size=196031 2024-03-04T17:50:21.405 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.404+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=QOID61CXZ2SQ tag=HSK48GHXP3IM type=write size=3341759 2024-03-04T17:50:21.410 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.408+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=6XU9NPD1KY8J type=del size=1762291 2024-03-04T17:50:21.414 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.412+0000 7fe960f568c0 -1 < failed to complete operation key=QGLN27WH1N2O tag=GW74Y83B0ZHS type=del size=2569776: (22) Invalid argument 2024-03-04T17:50:21.418 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.416+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=FHX0CA26MHOX type=del size=1179024 2024-03-04T17:50:21.425 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.424+0000 7fe960f568c0 1 < suggested remove operation key=8PZD01OOX3AF tag=8VJXZPXQCAKI type=del 2024-03-04T17:50:21.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.428+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=F9I96KCL81GY type=write size=202580 2024-03-04T17:50:21.436 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.436+0000 7fe960f568c0 1 < suggested update operation key=6X4ZV3BFZNDL tag=L2GSHT73OU80 type=del size=357252 2024-03-04T17:50:21.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.440+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~72MTG0IZAVBC.0 size=1048576 2024-03-04T17:50:21.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.444+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~72MTG0IZAVBC.1 size=223744 2024-03-04T17:50:21.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.444+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=72MTG0IZAVBC tag=XJCA1K7YAPC4 type=write size=2320896 2024-03-04T17:50:21.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.448+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~CKZVE1K5KK4O.0 size=1048576 2024-03-04T17:50:21.452 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.452+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~CKZVE1K5KK4O.1 size=366307 2024-03-04T17:50:21.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.456+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=CKZVE1K5KK4O tag=7ZWRWGDACQ7I type=write size=2463459 2024-03-04T17:50:21.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.464+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~FX40C2V3PXQ5.0 size=1048576 2024-03-04T17:50:21.467 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.468+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~FX40C2V3PXQ5.1 size=1048576 2024-03-04T17:50:21.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.468+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~FX40C2V3PXQ5.2 size=361957 2024-03-04T17:50:21.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.468+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=FX40C2V3PXQ5 tag=B11GRQKLRMGB type=write size=3507685 2024-03-04T17:50:21.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.476+0000 7fe960f568c0 1 < suggested update operation key=6X4ZV3BFZNDL tag=0YPNV4OMWUX7 type=del size=487212 2024-03-04T17:50:21.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.480+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=AV36UM1K3GE4 type=del size=620593 2024-03-04T17:50:21.488 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.488+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~KZPT64LNLQBT.0 size=518261 2024-03-04T17:50:21.494 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.492+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=KZPT64LNLQBT tag=6YMUSL589YVL type=write size=1566837 2024-03-04T17:50:21.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.500+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~PGL1F0GZQL7D.0 size=1048576 2024-03-04T17:50:21.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.504+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~PGL1F0GZQL7D.1 size=788509 2024-03-04T17:50:21.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.504+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=PGL1F0GZQL7D tag=9TGRN4GDW97Z type=write size=2885661 2024-03-04T17:50:21.509 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.508+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~6Z93VLJ2Y96P.0 size=350632 2024-03-04T17:50:21.516 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.516+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=6Z93VLJ2Y96P tag=DN5L09S9C6I7 type=write size=1399208 2024-03-04T17:50:21.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.520+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=SFSXM4ZI5TI0 type=del size=2154346 2024-03-04T17:50:21.524 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.524+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~JGUJDMXSNWX9.0 size=1048576 2024-03-04T17:50:21.527 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.528+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~JGUJDMXSNWX9.1 size=1048576 2024-03-04T17:50:21.531 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.528+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~JGUJDMXSNWX9.2 size=297692 2024-03-04T17:50:21.537 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.536+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=JGUJDMXSNWX9 tag=1NVWLC6ASLSX type=write size=3443420 2024-03-04T17:50:21.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.540+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=0TNNGKLZZCKG type=del size=93342 2024-03-04T17:50:21.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.548+0000 7fe960f568c0 1 < suggested remove operation key=2KZ4IFW54C5B tag=D40KBKV2QG5T type=del 2024-03-04T17:50:21.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.556+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~NPDH44M9E7MB.0 size=969239 2024-03-04T17:50:21.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.556+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=NPDH44M9E7MB tag=NRK31V22FN1Z type=write size=2017815 2024-03-04T17:50:21.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.556+0000 7fe960f568c0 -1 < failed to complete operation key=2KZ4IFW54C5B tag=ZGFW1TA9880D type=del size=2856070: (22) Invalid argument 2024-03-04T17:50:21.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.560+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=PDLXK59PCR55 type=del size=1042394 2024-03-04T17:50:21.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.564+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=RNTZ1ZP427F4 type=del 2024-03-04T17:50:21.573 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.572+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~ICEPCVA9TS7T.0 size=6898 2024-03-04T17:50:21.573 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.572+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=ICEPCVA9TS7T tag=UQ6ZZWWF8NVK type=write size=1055474 2024-03-04T17:50:21.577 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.576+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~7KE5EL45N9LT.0 size=289280 2024-03-04T17:50:21.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.580+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=7KE5EL45N9LT tag=WZQ3I03C2D5B type=write size=1337856 2024-03-04T17:50:21.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.584+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=12IA6W1WDNWM type=del size=3581559 2024-03-04T17:50:21.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.592+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~6D0V064QR0B2.0 size=1048576 2024-03-04T17:50:21.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.592+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~6D0V064QR0B2.1 size=1048576 2024-03-04T17:50:21.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.596+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~6D0V064QR0B2.2 size=750140 2024-03-04T17:50:21.604 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.604+0000 7fe960f568c0 1 < completed multipart upload key=U2G6YEVD5506 upload=6D0V064QR0B2 tag=PCSM3VBKL9UQ type=write size=3895868 2024-03-04T17:50:21.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.608+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=4IVGSOXLFERL type=del size=1802087 2024-03-04T17:50:21.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.612+0000 7fe960f568c0 -1 < failed to complete operation key=6X4ZV3BFZNDL tag=Z4QOTJJ2GLCS type=del size=2991514: (22) Invalid argument 2024-03-04T17:50:21.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.616+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=6KVNN417W83U type=del size=2945181 2024-03-04T17:50:21.620 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.620+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~WG8CM8YT13S0.0 size=1048576 2024-03-04T17:50:21.625 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.624+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~WG8CM8YT13S0.1 size=922461 2024-03-04T17:50:21.632 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.632+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=WG8CM8YT13S0 tag=30S6SCX7KQEJ type=write size=3019613 2024-03-04T17:50:21.636 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.636+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=ZZ6KKSUY49L3 type=write size=234521 2024-03-04T17:50:21.640 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.640+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=6XU9NPD1KY8J type=del 2024-03-04T17:50:21.647 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.644+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~7VHE7CSV18X5.0 size=776253 2024-03-04T17:50:21.647 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.644+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=7VHE7CSV18X5 tag=XNWSVB1O2Y1R type=write size=1824829 2024-03-04T17:50:21.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.648+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=FHX0CA26MHOX type=del 2024-03-04T17:50:21.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.652+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=O3LULTKF6Q0A type=del size=3874349 2024-03-04T17:50:21.658 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.656+0000 7fe960f568c0 1 < completed write operation key=QFF67JIW8Y8E tag=F9I96KCL81GY type=write size=202580 2024-03-04T17:50:21.662 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.660+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=2TZOD1G80331 type=del size=2322948 2024-03-04T17:50:21.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.664+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~72MTG0IZAVBC.0 size=1048576 2024-03-04T17:50:21.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.668+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~72MTG0IZAVBC.1 size=223744 2024-03-04T17:50:21.675 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.672+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=72MTG0IZAVBC tag=XJCA1K7YAPC4 type=write size=2320896 2024-03-04T17:50:21.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.680+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~245D7TSEK0SF.0 size=788917 2024-03-04T17:50:21.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.680+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=245D7TSEK0SF tag=COY10PCRRVQU type=write size=1837493 2024-03-04T17:50:21.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.684+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~FX40C2V3PXQ5.0 size=1048576 2024-03-04T17:50:21.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.688+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~FX40C2V3PXQ5.1 size=1048576 2024-03-04T17:50:21.691 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.692+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~FX40C2V3PXQ5.2 size=361957 2024-03-04T17:50:21.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.696+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=FX40C2V3PXQ5 tag=B11GRQKLRMGB type=write size=3507685 2024-03-04T17:50:21.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.704+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~3KXSHDF9H1MB.0 size=829636 2024-03-04T17:50:21.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.704+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=3KXSHDF9H1MB tag=CPCUG93UNAOX type=write size=1878212 2024-03-04T17:50:21.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.708+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=AV36UM1K3GE4 type=del 2024-03-04T17:50:21.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.716+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~JCM9V1Q5FZ6Q.0 size=900641 2024-03-04T17:50:21.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.716+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=JCM9V1Q5FZ6Q tag=C8UALQBHSSUQ type=write size=1949217 2024-03-04T17:50:21.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.720+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~PGL1F0GZQL7D.0 size=1048576 2024-03-04T17:50:21.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.720+0000 7fe960f568c0 1 < canceled part key=_multipart_GZ4RRIR5CC26.2~PGL1F0GZQL7D.1 size=788509 2024-03-04T17:50:21.725 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.724+0000 7fe960f568c0 1 < canceled multipart upload key=GZ4RRIR5CC26 upload=PGL1F0GZQL7D tag=9TGRN4GDW97Z type=write size=2885661 2024-03-04T17:50:21.729 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.728+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=HB0GNY6LRHY8 type=del size=679344 2024-03-04T17:50:21.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.732+0000 7fe960f568c0 -1 < failed to complete operation key=2KZ4IFW54C5B tag=SFSXM4ZI5TI0 type=del size=2154346: (22) Invalid argument 2024-03-04T17:50:21.737 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.736+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=E91HUY4JPN1M type=del size=1386307 2024-03-04T17:50:21.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.744+0000 7fe960f568c0 1 < suggested remove operation key=P28N6U79ZUV2 tag=0TNNGKLZZCKG type=del 2024-03-04T17:50:21.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.752+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~ZPLYZURAKPFV.0 size=1048576 2024-03-04T17:50:21.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.756+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~ZPLYZURAKPFV.1 size=1048576 2024-03-04T17:50:21.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.756+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~ZPLYZURAKPFV.2 size=190169 2024-03-04T17:50:21.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.756+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=ZPLYZURAKPFV tag=IT8U89L9HP4R type=write size=3335897 2024-03-04T17:50:21.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.760+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~NPDH44M9E7MB.0 size=969239 2024-03-04T17:50:21.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.768+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=NPDH44M9E7MB tag=NRK31V22FN1Z type=write size=2017815 2024-03-04T17:50:21.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.772+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=TJ5YNULS7D8U type=write size=843577 2024-03-04T17:50:21.776 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.776+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=PDLXK59PCR55 type=del 2024-03-04T17:50:21.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.780+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=47UZC8U22RR2 type=del size=1262447 2024-03-04T17:50:21.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.784+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~ICEPCVA9TS7T.0 size=6898 2024-03-04T17:50:21.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.788+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=ICEPCVA9TS7T tag=UQ6ZZWWF8NVK type=write size=1055474 2024-03-04T17:50:21.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.792+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=LN81L8OV7O1M type=del size=3459889 2024-03-04T17:50:21.798 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.796+0000 7fe960f568c0 1 < canceled operation key=ASC735ZCBLCG tag=12IA6W1WDNWM type=del size=3581559 2024-03-04T17:50:21.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.800+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=D69YMOTWE0C0 type=del size=2285022 2024-03-04T17:50:21.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.804+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=4IVGSOXLFERL type=del 2024-03-04T17:50:21.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.812+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~5ZBLUE41T73T.0 size=1048576 2024-03-04T17:50:21.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.816+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~5ZBLUE41T73T.1 size=1048576 2024-03-04T17:50:21.818 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.816+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~5ZBLUE41T73T.2 size=580496 2024-03-04T17:50:21.818 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.816+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=5ZBLUE41T73T tag=16D12COJTMPR type=write size=3726224 2024-03-04T17:50:21.826 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.824+0000 7fe960f568c0 1 < suggested update operation key=QFF67JIW8Y8E tag=6KVNN417W83U type=del size=2945181 2024-03-04T17:50:21.829 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.828+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=VH4JKELTECA0 type=write size=601602 2024-03-04T17:50:21.833 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.832+0000 7fe960f568c0 1 < completed write operation key=QGLN27WH1N2O tag=ZZ6KKSUY49L3 type=write size=234521 2024-03-04T17:50:21.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.840+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~7G20AR9UOTIK.0 size=1048576 2024-03-04T17:50:21.843 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.844+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~7G20AR9UOTIK.1 size=1048576 2024-03-04T17:50:21.846 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.844+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~7G20AR9UOTIK.2 size=710953 2024-03-04T17:50:21.846 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.844+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=7G20AR9UOTIK tag=IBFWUK19FL77 type=write size=3856681 2024-03-04T17:50:21.850 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.848+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~7VHE7CSV18X5.0 size=776253 2024-03-04T17:50:21.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.856+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=7VHE7CSV18X5 tag=XNWSVB1O2Y1R type=write size=1824829 2024-03-04T17:50:21.863 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.864+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~YQ1AKHF4MV3H.0 size=1048576 2024-03-04T17:50:21.867 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.864+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~YQ1AKHF4MV3H.1 size=1048576 2024-03-04T17:50:21.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.868+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~YQ1AKHF4MV3H.2 size=877744 2024-03-04T17:50:21.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.868+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=YQ1AKHF4MV3H tag=1V20H8YN1CK4 type=write size=4023472 2024-03-04T17:50:21.874 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.872+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=O3LULTKF6Q0A type=del 2024-03-04T17:50:21.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.876+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=NHIT9Q2C7P1B type=del size=168530 2024-03-04T17:50:21.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.880+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=2TZOD1G80331 type=del 2024-03-04T17:50:21.886 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.884+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=6LXPRNR0XHZY type=write size=40478 2024-03-04T17:50:21.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.888+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~245D7TSEK0SF.0 size=788917 2024-03-04T17:50:21.896 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.896+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=245D7TSEK0SF tag=COY10PCRRVQU type=write size=1837493 2024-03-04T17:50:21.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.904+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~Z611M04M1EP5.0 size=1048576 2024-03-04T17:50:21.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.904+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~Z611M04M1EP5.1 size=1048576 2024-03-04T17:50:21.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.908+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~Z611M04M1EP5.2 size=191163 2024-03-04T17:50:21.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.908+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=Z611M04M1EP5 tag=6D420VT1728C type=write size=3336891 2024-03-04T17:50:21.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.912+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~3KXSHDF9H1MB.0 size=829636 2024-03-04T17:50:21.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.916+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=3KXSHDF9H1MB tag=CPCUG93UNAOX type=write size=1878212 2024-03-04T17:50:21.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.920+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=RW93WFUS5351 type=del size=829188 2024-03-04T17:50:21.927 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.924+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~JCM9V1Q5FZ6Q.0 size=900641 2024-03-04T17:50:21.933 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.932+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=JCM9V1Q5FZ6Q tag=C8UALQBHSSUQ type=write size=1949217 2024-03-04T17:50:21.937 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.936+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=XMXEGFL9FU6U type=del size=254286 2024-03-04T17:50:21.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.940+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=HB0GNY6LRHY8 type=del 2024-03-04T17:50:21.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.944+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=7URE639IOXWK type=del size=593386 2024-03-04T17:50:21.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.952+0000 7fe960f568c0 1 < suggested update operation key=3YTAYUQLLUKY tag=E91HUY4JPN1M type=del size=1386307 2024-03-04T17:50:21.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.956+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=BXJW7OQ3VC05 type=del size=3075629 2024-03-04T17:50:21.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.960+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~ZPLYZURAKPFV.0 size=1048576 2024-03-04T17:50:21.964 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.964+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~ZPLYZURAKPFV.1 size=1048576 2024-03-04T17:50:21.967 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.964+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~ZPLYZURAKPFV.2 size=190169 2024-03-04T17:50:21.973 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.972+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=ZPLYZURAKPFV tag=IT8U89L9HP4R type=write size=3335897 2024-03-04T17:50:21.983 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.980+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~D2GX87V1WKCA.0 size=1048576 2024-03-04T17:50:21.985 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.984+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~D2GX87V1WKCA.1 size=1048576 2024-03-04T17:50:21.989 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.988+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~D2GX87V1WKCA.2 size=408576 2024-03-04T17:50:21.989 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.988+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=D2GX87V1WKCA tag=K3Y4XS41X532 type=write size=3554304 2024-03-04T17:50:21.992 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.992+0000 7fe960f568c0 1 < completed write operation key=UKZ079SVZZKG tag=TJ5YNULS7D8U type=write size=843577 2024-03-04T17:50:21.999 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.996+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~PJFY2ECUG7A1.0 size=80047 2024-03-04T17:50:21.999 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:21.996+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=PJFY2ECUG7A1 tag=XFA4TOVAYSXT type=write size=1128623 2024-03-04T17:50:22.003 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.000+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=47UZC8U22RR2 type=del 2024-03-04T17:50:22.009 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.008+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~4SKS5NT3ITLU.0 size=1048576 2024-03-04T17:50:22.012 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.012+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~4SKS5NT3ITLU.1 size=267103 2024-03-04T17:50:22.012 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.012+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=4SKS5NT3ITLU tag=XCWD5F3N9IL2 type=write size=2364255 2024-03-04T17:50:22.016 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.016+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=LN81L8OV7O1M type=del 2024-03-04T17:50:22.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.020+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=AZGVGEXZ7W4V type=del size=825761 2024-03-04T17:50:22.024 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.024+0000 7fe960f568c0 -1 < failed to complete operation key=3YTAYUQLLUKY tag=D69YMOTWE0C0 type=del size=2285022: (22) Invalid argument 2024-03-04T17:50:22.028 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.028+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=Y429K0PVAWW3 type=del size=1367641 2024-03-04T17:50:22.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.032+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~5ZBLUE41T73T.0 size=1048576 2024-03-04T17:50:22.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.036+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~5ZBLUE41T73T.1 size=1048576 2024-03-04T17:50:22.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.036+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~5ZBLUE41T73T.2 size=580496 2024-03-04T17:50:22.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.044+0000 7fe960f568c0 1 < completed multipart upload key=ASC735ZCBLCG upload=5ZBLUE41T73T tag=16D12COJTMPR type=write size=3726224 2024-03-04T17:50:22.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.048+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=SLZEKT0PY36C type=write size=138833 2024-03-04T17:50:22.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.056+0000 7fe960f568c0 1 < suggested remove operation key=WNXVI6K5MU3R tag=VH4JKELTECA0 type=write 2024-03-04T17:50:22.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.064+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~N25XZCY85ZE0.0 size=1048576 2024-03-04T17:50:22.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.068+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~N25XZCY85ZE0.1 size=819398 2024-03-04T17:50:22.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.068+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=N25XZCY85ZE0 tag=JRYGDUHTV3RR type=write size=2916550 2024-03-04T17:50:22.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.068+0000 7fe960f568c0 1 < canceled part key=_multipart_NF4OIJWJ83DE.2~7G20AR9UOTIK.0 size=1048576 2024-03-04T17:50:22.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.072+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~7G20AR9UOTIK.1 size=1048576 2024-03-04T17:50:22.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.076+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~7G20AR9UOTIK.2 size=710953 2024-03-04T17:50:22.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.080+0000 7fe960f568c0 1 < canceled multipart upload key=NF4OIJWJ83DE upload=7G20AR9UOTIK tag=IBFWUK19FL77 type=write size=3856681 2024-03-04T17:50:22.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.088+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~U8VG027DPTER.0 size=194964 2024-03-04T17:50:22.091 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.088+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=U8VG027DPTER tag=QO60X8L0HBEF type=write size=1243540 2024-03-04T17:50:22.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.096+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~YQ1AKHF4MV3H.0 size=1048576 2024-03-04T17:50:22.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.096+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~YQ1AKHF4MV3H.1 size=1048576 2024-03-04T17:50:22.102 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.100+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~YQ1AKHF4MV3H.2 size=877744 2024-03-04T17:50:22.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.108+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=YQ1AKHF4MV3H tag=1V20H8YN1CK4 type=write size=4023472 2024-03-04T17:50:22.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.112+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=IYOFMAREOMWA type=del size=1967023 2024-03-04T17:50:22.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.116+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=NHIT9Q2C7P1B type=del 2024-03-04T17:50:22.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.120+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=2HQXZG60435L type=del size=1391337 2024-03-04T17:50:22.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.124+0000 7fe960f568c0 1 < completed write operation key=BFF9OD00TU54 tag=6LXPRNR0XHZY type=write size=40478 2024-03-04T17:50:22.128 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.128+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=E5GP0R6QC7CM type=del size=2569512 2024-03-04T17:50:22.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.132+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~Z611M04M1EP5.0 size=1048576 2024-03-04T17:50:22.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.132+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~Z611M04M1EP5.1 size=1048576 2024-03-04T17:50:22.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.136+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~Z611M04M1EP5.2 size=191163 2024-03-04T17:50:22.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.144+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=Z611M04M1EP5 tag=6D420VT1728C type=write size=3336891 2024-03-04T17:50:22.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.144+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=T4U63362FUO0 type=del size=205406 2024-03-04T17:50:22.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.148+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=RW93WFUS5351 type=del 2024-03-04T17:50:22.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.152+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=5ZR5CU509FCV type=del size=907092 2024-03-04T17:50:22.159 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.156+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=XMXEGFL9FU6U type=del 2024-03-04T17:50:22.166 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.164+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~OY1GWMDHH5WQ.0 size=362286 2024-03-04T17:50:22.166 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.164+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=OY1GWMDHH5WQ tag=EQCLKEL03KQI type=write size=1410862 2024-03-04T17:50:22.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.168+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=7URE639IOXWK type=del 2024-03-04T17:50:22.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.172+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=BFUSQ0QY6PJ5 type=del size=2190814 2024-03-04T17:50:22.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.176+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=BXJW7OQ3VC05 type=del 2024-03-04T17:50:22.183 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.184+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=IERD7AMVXUIX type=del size=2345897 2024-03-04T17:50:22.188 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.188+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~D2GX87V1WKCA.0 size=1048576 2024-03-04T17:50:22.191 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.188+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~D2GX87V1WKCA.1 size=1048576 2024-03-04T17:50:22.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.192+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~D2GX87V1WKCA.2 size=408576 2024-03-04T17:50:22.201 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.200+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=D2GX87V1WKCA tag=K3Y4XS41X532 type=write size=3554304 2024-03-04T17:50:22.205 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.204+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=DCPB2L4CHHN9 type=del size=497826 2024-03-04T17:50:22.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.208+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~PJFY2ECUG7A1.0 size=80047 2024-03-04T17:50:22.215 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.212+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=PJFY2ECUG7A1 tag=XFA4TOVAYSXT type=write size=1128623 2024-03-04T17:50:22.222 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.220+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~47NRES3TOPTQ.0 size=6856 2024-03-04T17:50:22.222 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.220+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=47NRES3TOPTQ tag=3JBQID02843K type=write size=1055432 2024-03-04T17:50:22.226 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.224+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~4SKS5NT3ITLU.0 size=1048576 2024-03-04T17:50:22.229 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.228+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~4SKS5NT3ITLU.1 size=267103 2024-03-04T17:50:22.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.236+0000 7fe960f568c0 1 < completed multipart upload key=2MKQA5JYWU8L upload=4SKS5NT3ITLU tag=XCWD5F3N9IL2 type=write size=2364255 2024-03-04T17:50:22.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.240+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=OX8XHBNTHU6T type=del size=2991455 2024-03-04T17:50:22.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.244+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=AZGVGEXZ7W4V type=del 2024-03-04T17:50:22.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.248+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=VHOV4RRVIBYY type=del size=2783854 2024-03-04T17:50:22.252 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.252+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=Y429K0PVAWW3 type=del 2024-03-04T17:50:22.260 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.260+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~BVYRT2EF3ZKQ.0 size=1048576 2024-03-04T17:50:22.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.260+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~BVYRT2EF3ZKQ.1 size=1048576 2024-03-04T17:50:22.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.264+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~BVYRT2EF3ZKQ.2 size=772998 2024-03-04T17:50:22.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.264+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=BVYRT2EF3ZKQ tag=XOTPTXGONTF7 type=write size=3918726 2024-03-04T17:50:22.270 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.268+0000 7fe960f568c0 -1 < failed to complete operation key=WNXVI6K5MU3R tag=SLZEKT0PY36C type=write size=138833: (22) Invalid argument 2024-03-04T17:50:22.274 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.272+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=9W30X4HO3Q2X type=write size=818167 2024-03-04T17:50:22.275 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.272+0000 7fe960f568c0 1 < canceled part key=_multipart_XSQ3GE2UHMZG.2~N25XZCY85ZE0.0 size=1048576 2024-03-04T17:50:22.275 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.272+0000 7fe960f568c0 1 < canceled part key=_multipart_XSQ3GE2UHMZG.2~N25XZCY85ZE0.1 size=819398 2024-03-04T17:50:22.281 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.280+0000 7fe960f568c0 1 < canceled multipart upload key=XSQ3GE2UHMZG upload=N25XZCY85ZE0 tag=JRYGDUHTV3RR type=write size=2916550 2024-03-04T17:50:22.285 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.284+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=7O3LHERGWB8G type=del size=1648726 2024-03-04T17:50:22.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.288+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~U8VG027DPTER.0 size=194964 2024-03-04T17:50:22.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.296+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=U8VG027DPTER tag=QO60X8L0HBEF type=write size=1243540 2024-03-04T17:50:22.300 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.300+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=RTIF3UW1WEI4 type=write size=1037211 2024-03-04T17:50:22.304 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.304+0000 7fe960f568c0 1 < canceled operation key=JP3FRB6MD7B4 tag=IYOFMAREOMWA type=del size=1967023 2024-03-04T17:50:22.311 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.312+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~0PRJOG7EWIIV.0 size=1048576 2024-03-04T17:50:22.314 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.312+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~0PRJOG7EWIIV.1 size=1048576 2024-03-04T17:50:22.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.316+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~0PRJOG7EWIIV.2 size=556781 2024-03-04T17:50:22.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.316+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=0PRJOG7EWIIV tag=NCZ6FIOCI99C type=write size=3702509 2024-03-04T17:50:22.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.320+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=2HQXZG60435L type=del 2024-03-04T17:50:22.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.324+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=NZNCI1O59LFB type=write size=357216 2024-03-04T17:50:22.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.328+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=E5GP0R6QC7CM type=del 2024-03-04T17:50:22.337 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.336+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~5A2B91WJ71PP.0 size=1048576 2024-03-04T17:50:22.339 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.340+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~5A2B91WJ71PP.1 size=1048576 2024-03-04T17:50:22.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.344+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~5A2B91WJ71PP.2 size=690143 2024-03-04T17:50:22.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.344+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=5A2B91WJ71PP tag=04H8JDZ76ZDO type=write size=3835871 2024-03-04T17:50:22.350 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.348+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=T4U63362FUO0 type=del 2024-03-04T17:50:22.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.352+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=AHSW6W7QL839 type=del size=3270337 2024-03-04T17:50:22.358 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.356+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=5ZR5CU509FCV type=del 2024-03-04T17:50:22.362 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.360+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=ZSB5HYZE0LR1 type=del size=866814 2024-03-04T17:50:22.367 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.368+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~OY1GWMDHH5WQ.0 size=362286 2024-03-04T17:50:22.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.372+0000 7fe960f568c0 1 < completed multipart upload key=U2G6YEVD5506 upload=OY1GWMDHH5WQ tag=EQCLKEL03KQI type=write size=1410862 2024-03-04T17:50:22.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.376+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=71FL3D68H6SN type=del size=3236416 2024-03-04T17:50:22.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.380+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=BFUSQ0QY6PJ5 type=del 2024-03-04T17:50:22.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.384+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=ED1YIP81UB0Z type=del size=1839929 2024-03-04T17:50:22.391 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.388+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=IERD7AMVXUIX type=del 2024-03-04T17:50:22.397 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.396+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~NEVW837MO97O.0 size=1048576 2024-03-04T17:50:22.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.400+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~NEVW837MO97O.1 size=401667 2024-03-04T17:50:22.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.400+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=NEVW837MO97O tag=FFQEZZEYLZ12 type=write size=2498819 2024-03-04T17:50:22.405 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.404+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=DCPB2L4CHHN9 type=del 2024-03-04T17:50:22.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.412+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~Q01INCKP3BB9.0 size=420867 2024-03-04T17:50:22.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.412+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=Q01INCKP3BB9 tag=B0YSY1JYZYFH type=write size=1469443 2024-03-04T17:50:22.417 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.416+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~47NRES3TOPTQ.0 size=6856 2024-03-04T17:50:22.423 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.424+0000 7fe960f568c0 1 < completed multipart upload key=ASC735ZCBLCG upload=47NRES3TOPTQ tag=3JBQID02843K type=write size=1055432 2024-03-04T17:50:22.430 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.428+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~YVZ00IHAH3YJ.0 size=1048576 2024-03-04T17:50:22.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.432+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~YVZ00IHAH3YJ.1 size=1048576 2024-03-04T17:50:22.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.436+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~YVZ00IHAH3YJ.2 size=812732 2024-03-04T17:50:22.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.436+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=YVZ00IHAH3YJ tag=NIF37C3WCW02 type=write size=3958460 2024-03-04T17:50:22.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.440+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=OX8XHBNTHU6T type=del 2024-03-04T17:50:22.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.444+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~Q01B0SH82VKV.0 size=1048576 2024-03-04T17:50:22.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.448+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~Q01B0SH82VKV.1 size=1048576 2024-03-04T17:50:22.452 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.452+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~Q01B0SH82VKV.2 size=47515 2024-03-04T17:50:22.452 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.452+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=Q01B0SH82VKV tag=CX3J0U6JTKSM type=write size=3193243 2024-03-04T17:50:22.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.456+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=VHOV4RRVIBYY type=del 2024-03-04T17:50:22.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.460+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=NT479ZRZJ26Z type=del size=2575068 2024-03-04T17:50:22.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.464+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~BVYRT2EF3ZKQ.0 size=1048576 2024-03-04T17:50:22.468 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.468+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~BVYRT2EF3ZKQ.1 size=1048576 2024-03-04T17:50:22.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.472+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~BVYRT2EF3ZKQ.2 size=772998 2024-03-04T17:50:22.478 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.476+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=BVYRT2EF3ZKQ tag=XOTPTXGONTF7 type=write size=3918726 2024-03-04T17:50:22.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.480+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=V2NWZA2PC1EF type=del size=3204571 2024-03-04T17:50:22.486 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.484+0000 7fe960f568c0 1 < completed write operation key=JT02WL9JYTNM tag=9W30X4HO3Q2X type=write size=818167 2024-03-04T17:50:22.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.492+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=AWDPVLOOQGKN type=del size=943799 2024-03-04T17:50:22.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.496+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=7O3LHERGWB8G type=del 2024-03-04T17:50:22.503 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.500+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~Q9Y4C1R55KWH.0 size=575067 2024-03-04T17:50:22.503 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.500+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=Q9Y4C1R55KWH tag=G2L0YNGRB1YZ type=write size=1623643 2024-03-04T17:50:22.507 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.504+0000 7fe960f568c0 1 < completed write operation key=KYSLL05AUHAC tag=RTIF3UW1WEI4 type=write size=1037211 2024-03-04T17:50:22.513 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.512+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~82R1SAUK5YX8.0 size=1048576 2024-03-04T17:50:22.516 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.516+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~82R1SAUK5YX8.1 size=1048576 2024-03-04T17:50:22.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.516+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~82R1SAUK5YX8.2 size=11361 2024-03-04T17:50:22.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.516+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=82R1SAUK5YX8 tag=T7DSGS7GQKRQ type=write size=3157089 2024-03-04T17:50:22.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.520+0000 7fe960f568c0 1 < canceled part key=_multipart_QGLN27WH1N2O.2~0PRJOG7EWIIV.0 size=1048576 2024-03-04T17:50:22.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.524+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~0PRJOG7EWIIV.1 size=1048576 2024-03-04T17:50:22.527 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.524+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~0PRJOG7EWIIV.2 size=556781 2024-03-04T17:50:22.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.536+0000 7fe960f568c0 1 < canceled multipart upload key=QGLN27WH1N2O upload=0PRJOG7EWIIV tag=NCZ6FIOCI99C type=write size=3702509 2024-03-04T17:50:22.537 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.536+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=ELT2QNCCDE7N type=write size=381974 2024-03-04T17:50:22.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.540+0000 7fe960f568c0 1 < completed write operation key=JSO6WIYMPCF6 tag=NZNCI1O59LFB type=write size=357216 2024-03-04T17:50:22.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.544+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=1Q77WTLGV3F6 type=del size=2605602 2024-03-04T17:50:22.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.548+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~5A2B91WJ71PP.0 size=1048576 2024-03-04T17:50:22.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.552+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~5A2B91WJ71PP.1 size=1048576 2024-03-04T17:50:22.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.556+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~5A2B91WJ71PP.2 size=690143 2024-03-04T17:50:22.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.560+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=5A2B91WJ71PP tag=04H8JDZ76ZDO type=write size=3835871 2024-03-04T17:50:22.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.564+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=0MK2LB73T0KG type=del size=330956 2024-03-04T17:50:22.570 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.568+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=AHSW6W7QL839 type=del 2024-03-04T17:50:22.574 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.572+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=NSFIMZWGUPHQ type=del size=2447512 2024-03-04T17:50:22.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.580+0000 7fe960f568c0 1 < suggested remove operation key=7Z7BPZ8XGBHZ tag=ZSB5HYZE0LR1 type=del 2024-03-04T17:50:22.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.584+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=VFDJBEXOUSMO type=del size=2855542 2024-03-04T17:50:22.590 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.588+0000 7fe960f568c0 1 < canceled operation key=KYSLL05AUHAC tag=71FL3D68H6SN type=del size=3236416 2024-03-04T17:50:22.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.592+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=WRJOD0PDWWE0 type=del size=1495858 2024-03-04T17:50:22.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.596+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=ED1YIP81UB0Z type=del 2024-03-04T17:50:22.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.600+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=QDCUKK15O4ID type=del size=2597045 2024-03-04T17:50:22.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.604+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~NEVW837MO97O.0 size=1048576 2024-03-04T17:50:22.609 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.608+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~NEVW837MO97O.1 size=401667 2024-03-04T17:50:22.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.616+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=NEVW837MO97O tag=FFQEZZEYLZ12 type=write size=2498819 2024-03-04T17:50:22.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.620+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=HS1LZJO6F6B7 type=del size=2897729 2024-03-04T17:50:22.624 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.624+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~Q01INCKP3BB9.0 size=420867 2024-03-04T17:50:22.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.628+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=Q01INCKP3BB9 tag=B0YSY1JYZYFH type=write size=1469443 2024-03-04T17:50:22.635 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.632+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=54WM67IRWBY3 type=del size=2415955 2024-03-04T17:50:22.636 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.636+0000 7fe960f568c0 1 < canceled part key=_multipart_3X4719GHK3BI.2~YVZ00IHAH3YJ.0 size=1048576 2024-03-04T17:50:22.639 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.636+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~YVZ00IHAH3YJ.1 size=1048576 2024-03-04T17:50:22.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.640+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~YVZ00IHAH3YJ.2 size=812732 2024-03-04T17:50:22.648 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.648+0000 7fe960f568c0 1 < canceled multipart upload key=3X4719GHK3BI upload=YVZ00IHAH3YJ tag=NIF37C3WCW02 type=write size=3958460 2024-03-04T17:50:22.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.652+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=Q3FOMXN0CT5X type=del size=2904898 2024-03-04T17:50:22.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.656+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~Q01B0SH82VKV.0 size=1048576 2024-03-04T17:50:22.660 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.660+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~Q01B0SH82VKV.1 size=1048576 2024-03-04T17:50:22.663 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.660+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~Q01B0SH82VKV.2 size=47515 2024-03-04T17:50:22.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.668+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=Q01B0SH82VKV tag=CX3J0U6JTKSM type=write size=3193243 2024-03-04T17:50:22.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.676+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~J4Z92HVZRMLV.0 size=736104 2024-03-04T17:50:22.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.676+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=J4Z92HVZRMLV tag=I406JDSEDFTU type=write size=1784680 2024-03-04T17:50:22.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.680+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=NT479ZRZJ26Z type=del 2024-03-04T17:50:22.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.688+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~5Y2SF08CAD7E.0 size=1048576 2024-03-04T17:50:22.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.688+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~5Y2SF08CAD7E.1 size=944062 2024-03-04T17:50:22.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.688+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=5Y2SF08CAD7E tag=YXZRZWHK3DB4 type=write size=3041214 2024-03-04T17:50:22.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.692+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=V2NWZA2PC1EF type=del 2024-03-04T17:50:22.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.696+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=HMA80GP8GYU1 type=del size=922182 2024-03-04T17:50:22.702 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.700+0000 7fe960f568c0 1 < canceled operation key=QGLN27WH1N2O tag=AWDPVLOOQGKN type=del size=943799 2024-03-04T17:50:22.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.704+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=QSRXIWCJV0SV type=write size=280191 2024-03-04T17:50:22.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.708+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~Q9Y4C1R55KWH.0 size=575067 2024-03-04T17:50:22.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.720+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=Q9Y4C1R55KWH tag=G2L0YNGRB1YZ type=write size=1623643 2024-03-04T17:50:22.726 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.724+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~5DSFKR4ZW356.0 size=58659 2024-03-04T17:50:22.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.724+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=5DSFKR4ZW356 tag=SJVASCQE0JOO type=write size=1107235 2024-03-04T17:50:22.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.728+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~82R1SAUK5YX8.0 size=1048576 2024-03-04T17:50:22.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.732+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~82R1SAUK5YX8.1 size=1048576 2024-03-04T17:50:22.736 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.736+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~82R1SAUK5YX8.2 size=11361 2024-03-04T17:50:22.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.740+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=82R1SAUK5YX8 tag=T7DSGS7GQKRQ type=write size=3157089 2024-03-04T17:50:22.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.748+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=8612G22KRUM2 type=write size=456258 2024-03-04T17:50:22.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.752+0000 7fe960f568c0 1 < completed write operation key=78N8HHC4O4CG tag=ELT2QNCCDE7N type=write size=381974 2024-03-04T17:50:22.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.756+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=VWUC40LGZUAR type=del size=1492084 2024-03-04T17:50:22.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.760+0000 7fe960f568c0 1 < suggested remove operation key=2FUTMG0SSXBY tag=1Q77WTLGV3F6 type=del 2024-03-04T17:50:22.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.764+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=Y04WFFVN4U8R type=del size=236377 2024-03-04T17:50:22.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.768+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=0MK2LB73T0KG type=del 2024-03-04T17:50:22.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.772+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=56NTZL6L7NF9 type=write size=100839 2024-03-04T17:50:22.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.780+0000 7fe960f568c0 1 < suggested remove operation key=NF4OIJWJ83DE tag=NSFIMZWGUPHQ type=del 2024-03-04T17:50:22.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.784+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=ONVKT8HQELRH type=del size=222077 2024-03-04T17:50:22.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.792+0000 7fe960f568c0 1 < suggested update operation key=3YTAYUQLLUKY tag=VFDJBEXOUSMO type=del size=2855542 2024-03-04T17:50:22.798 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.796+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=UEL05WLIBPZM type=del size=1412433 2024-03-04T17:50:22.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.800+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=WRJOD0PDWWE0 type=del 2024-03-04T17:50:22.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.804+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=AAP9USS330KF type=del size=1728822 2024-03-04T17:50:22.809 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.808+0000 7fe960f568c0 -1 < failed to complete operation key=3YTAYUQLLUKY tag=QDCUKK15O4ID type=del size=2597045: (22) Invalid argument 2024-03-04T17:50:22.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.812+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=0SHQK96UPAI4 type=del size=1480586 2024-03-04T17:50:22.822 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.820+0000 7fe960f568c0 1 < suggested remove operation key=GZ4RRIR5CC26 tag=HS1LZJO6F6B7 type=del 2024-03-04T17:50:22.829 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.828+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~I5SK71R06HX4.0 size=1048576 2024-03-04T17:50:22.833 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.836+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~I5SK71R06HX4.1 size=442942 2024-03-04T17:50:22.833 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.836+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=I5SK71R06HX4 tag=K9ELL4SWUMHD type=write size=2540094 2024-03-04T17:50:22.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.836+0000 7fe960f568c0 -1 < failed to complete operation key=2FUTMG0SSXBY tag=54WM67IRWBY3 type=del size=2415955: (22) Invalid argument 2024-03-04T17:50:22.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.840+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=PDDKKAJZ9YHR type=del size=24718 2024-03-04T17:50:22.845 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.844+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=Q3FOMXN0CT5X type=del 2024-03-04T17:50:22.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.848+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=JSQGQXKV0RXX type=write size=137040 2024-03-04T17:50:22.850 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.848+0000 7fe960f568c0 1 < canceled part key=_multipart_ASC735ZCBLCG.2~J4Z92HVZRMLV.0 size=736104 2024-03-04T17:50:22.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.856+0000 7fe960f568c0 1 < canceled multipart upload key=ASC735ZCBLCG upload=J4Z92HVZRMLV tag=I406JDSEDFTU type=write size=1784680 2024-03-04T17:50:22.860 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.860+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=QUT9EEGAVYLD type=del size=535991 2024-03-04T17:50:22.864 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.864+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~5Y2SF08CAD7E.0 size=1048576 2024-03-04T17:50:22.866 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.864+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~5Y2SF08CAD7E.1 size=944062 2024-03-04T17:50:22.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.872+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=5Y2SF08CAD7E tag=YXZRZWHK3DB4 type=write size=3041214 2024-03-04T17:50:22.876 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.876+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=6D80N4G99WQ6 type=del size=4089494 2024-03-04T17:50:22.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.880+0000 7fe960f568c0 1 < canceled operation key=WNXVI6K5MU3R tag=HMA80GP8GYU1 type=del size=922182 2024-03-04T17:50:22.885 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.884+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=FO38YKLBSBCC type=del size=2624210 2024-03-04T17:50:22.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.888+0000 7fe960f568c0 1 < completed write operation key=KYSLL05AUHAC tag=QSRXIWCJV0SV type=write size=280191 2024-03-04T17:50:22.894 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.892+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=GNF6JJ27L7VK type=del size=768654 2024-03-04T17:50:22.895 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.892+0000 7fe960f568c0 1 < canceled part key=_multipart_R6ZDVOUE9NXJ.2~5DSFKR4ZW356.0 size=58659 2024-03-04T17:50:22.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.900+0000 7fe960f568c0 1 < canceled multipart upload key=R6ZDVOUE9NXJ upload=5DSFKR4ZW356 tag=SJVASCQE0JOO type=write size=1107235 2024-03-04T17:50:22.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.904+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=REP472NLGNWF type=write size=270649 2024-03-04T17:50:22.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.908+0000 7fe960f568c0 1 < completed write operation key=78N8HHC4O4CG tag=8612G22KRUM2 type=write size=456258 2024-03-04T17:50:22.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.912+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=QFN5T6OPJKD0 type=del size=1936127 2024-03-04T17:50:22.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.924+0000 7fe960f568c0 1 < suggested update operation key=2KZ4IFW54C5B tag=VWUC40LGZUAR type=del size=1492084 2024-03-04T17:50:22.928 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.932+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=5C1CI6NZ861M type=del size=1151583 2024-03-04T17:50:22.933 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.932+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=Y04WFFVN4U8R type=del 2024-03-04T17:50:22.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.940+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~QX0DYBBQ8PV2.0 size=1048576 2024-03-04T17:50:22.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.944+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~QX0DYBBQ8PV2.1 size=1048576 2024-03-04T17:50:22.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.944+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~QX0DYBBQ8PV2.2 size=212353 2024-03-04T17:50:22.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.944+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=QX0DYBBQ8PV2 tag=Z3OF1QP9GY5V type=write size=3358081 2024-03-04T17:50:22.950 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.948+0000 7fe960f568c0 -1 < failed to complete operation key=GZ4RRIR5CC26 tag=56NTZL6L7NF9 type=write size=100839: (22) Invalid argument 2024-03-04T17:50:22.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.952+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=5OYMHCL8EHL6 type=del size=1670655 2024-03-04T17:50:22.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.956+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=ONVKT8HQELRH type=del 2024-03-04T17:50:22.964 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.964+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~WHKGQ6ICYNBA.0 size=1048576 2024-03-04T17:50:22.967 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.968+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~WHKGQ6ICYNBA.1 size=873630 2024-03-04T17:50:22.967 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.968+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=WHKGQ6ICYNBA tag=D62WO98EQ5RT type=write size=2970782 2024-03-04T17:50:22.971 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.972+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=UEL05WLIBPZM type=del 2024-03-04T17:50:22.975 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.972+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=WB30713LQI1V type=del size=4044940 2024-03-04T17:50:22.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.976+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=AAP9USS330KF type=del 2024-03-04T17:50:22.983 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.984+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=KN0CFTJT1PC8 type=del size=3138462 2024-03-04T17:50:22.987 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.988+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=0SHQK96UPAI4 type=del 2024-03-04T17:50:22.994 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.992+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~0Z96N1OYNXMV.0 size=294625 2024-03-04T17:50:22.995 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.992+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=0Z96N1OYNXMV tag=LW7LO6HG51XN type=write size=1343201 2024-03-04T17:50:22.998 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.996+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~I5SK71R06HX4.0 size=1048576 2024-03-04T17:50:22.998 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:22.996+0000 7fe960f568c0 1 < canceled part key=_multipart_RII14ZGX2EL1.2~I5SK71R06HX4.1 size=442942 2024-03-04T17:50:23.005 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.008+0000 7fe960f568c0 1 < canceled multipart upload key=RII14ZGX2EL1 upload=I5SK71R06HX4 tag=K9ELL4SWUMHD type=write size=2540094 2024-03-04T17:50:23.010 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.008+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=EILYCU8CK2JG type=del size=3855982 2024-03-04T17:50:23.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.016+0000 7fe960f568c0 1 < suggested update operation key=ASC735ZCBLCG tag=PDDKKAJZ9YHR type=del size=24718 2024-03-04T17:50:23.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.020+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=RTPEON9PLY2A type=write size=108003 2024-03-04T17:50:23.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.024+0000 7fe960f568c0 1 < completed write operation key=FLYDZHOIWO6L tag=JSQGQXKV0RXX type=write size=137040 2024-03-04T17:50:23.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.032+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~0F4R1WY8T1YF.0 size=1048576 2024-03-04T17:50:23.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.032+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~0F4R1WY8T1YF.1 size=511437 2024-03-04T17:50:23.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.032+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=0F4R1WY8T1YF tag=ZNJLRBPYG0VE type=write size=2608589 2024-03-04T17:50:23.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.036+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=QUT9EEGAVYLD type=del 2024-03-04T17:50:23.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.044+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~R6MMON1SHQT1.0 size=158125 2024-03-04T17:50:23.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.044+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=R6MMON1SHQT1 tag=N43EP4RBDP6L type=write size=1206701 2024-03-04T17:50:23.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.048+0000 7fe960f568c0 -1 < failed to complete operation key=2KZ4IFW54C5B tag=6D80N4G99WQ6 type=del size=4089494: (22) Invalid argument 2024-03-04T17:50:23.056 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.056+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~L0N2F5RUZAVQ.0 size=1048576 2024-03-04T17:50:23.059 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.060+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~L0N2F5RUZAVQ.1 size=870668 2024-03-04T17:50:23.059 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.060+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=L0N2F5RUZAVQ tag=M14UJ3DOMW7P type=write size=2967820 2024-03-04T17:50:23.067 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.064+0000 7fe960f568c0 1 < suggested update operation key=NF4OIJWJ83DE tag=FO38YKLBSBCC type=del size=2624210 2024-03-04T17:50:23.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.068+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=4ZPN7UA2YELH type=del size=1055438 2024-03-04T17:50:23.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.072+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=GNF6JJ27L7VK type=del 2024-03-04T17:50:23.078 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.076+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=7XS8DCG81PLR type=del size=2143887 2024-03-04T17:50:23.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.080+0000 7fe960f568c0 1 < completed write operation key=FLYDZHOIWO6L tag=REP472NLGNWF type=write size=270649 2024-03-04T17:50:23.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.084+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=RJEE7ORV17VB type=del size=1916268 2024-03-04T17:50:23.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.088+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=QFN5T6OPJKD0 type=del 2024-03-04T17:50:23.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.096+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=M3GE8RREBXGR type=del size=4130628 2024-03-04T17:50:23.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.100+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=5C1CI6NZ861M type=del 2024-03-04T17:50:23.103 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.104+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=0DN1N9X4ZYZV type=del size=1219106 2024-03-04T17:50:23.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.108+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~QX0DYBBQ8PV2.0 size=1048576 2024-03-04T17:50:23.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.108+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~QX0DYBBQ8PV2.1 size=1048576 2024-03-04T17:50:23.113 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.112+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~QX0DYBBQ8PV2.2 size=212353 2024-03-04T17:50:23.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.120+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=QX0DYBBQ8PV2 tag=Z3OF1QP9GY5V type=write size=3358081 2024-03-04T17:50:23.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.124+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=2Z0PB7N0X2K6 type=del size=3724180 2024-03-04T17:50:23.128 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.128+0000 7fe960f568c0 1 < canceled operation key=QGLN27WH1N2O tag=5OYMHCL8EHL6 type=del size=1670655 2024-03-04T17:50:23.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.136+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=521RYGA0NBO4 type=del size=1729860 2024-03-04T17:50:23.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.136+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~WHKGQ6ICYNBA.0 size=1048576 2024-03-04T17:50:23.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.136+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~WHKGQ6ICYNBA.1 size=873630 2024-03-04T17:50:23.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.144+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=WHKGQ6ICYNBA tag=D62WO98EQ5RT type=write size=2970782 2024-03-04T17:50:23.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.148+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=QXJ35TFEY786 type=del size=3112782 2024-03-04T17:50:23.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.152+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=WB30713LQI1V type=del 2024-03-04T17:50:23.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.156+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~B3S6FLRJKJWU.0 size=1048576 2024-03-04T17:50:23.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.160+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~B3S6FLRJKJWU.1 size=146204 2024-03-04T17:50:23.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.160+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=B3S6FLRJKJWU tag=17B0PJ9GSRIC type=write size=2243356 2024-03-04T17:50:23.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.164+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=KN0CFTJT1PC8 type=del 2024-03-04T17:50:23.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.172+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~DNKQEXUQHW94.0 size=448027 2024-03-04T17:50:23.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.172+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=DNKQEXUQHW94 tag=X92EK56RMDTE type=write size=1496603 2024-03-04T17:50:23.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.176+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~0Z96N1OYNXMV.0 size=294625 2024-03-04T17:50:23.182 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.180+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=0Z96N1OYNXMV tag=LW7LO6HG51XN type=write size=1343201 2024-03-04T17:50:23.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.184+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=T0XE6SCPMXU2 type=del size=4051900 2024-03-04T17:50:23.190 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.188+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=EILYCU8CK2JG type=del 2024-03-04T17:50:23.197 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.196+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~DVM88MNWYC1U.0 size=1048576 2024-03-04T17:50:23.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.200+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~DVM88MNWYC1U.1 size=405252 2024-03-04T17:50:23.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.200+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=DVM88MNWYC1U tag=QSIIKQ2ETL35 type=write size=2502404 2024-03-04T17:50:23.204 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.204+0000 7fe960f568c0 1 < completed write operation key=8PZD01OOX3AF tag=RTPEON9PLY2A type=write size=108003 2024-03-04T17:50:23.207 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.208+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=S0PONH9D3J9Y type=del size=3444293 2024-03-04T17:50:23.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.212+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~0F4R1WY8T1YF.0 size=1048576 2024-03-04T17:50:23.215 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.216+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~0F4R1WY8T1YF.1 size=511437 2024-03-04T17:50:23.221 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.220+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=0F4R1WY8T1YF tag=ZNJLRBPYG0VE type=write size=2608589 2024-03-04T17:50:23.225 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.224+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=4VVI6N3CN1U7 type=del size=1894358 2024-03-04T17:50:23.226 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.224+0000 7fe960f568c0 1 < canceled part key=_multipart_P28N6U79ZUV2.2~R6MMON1SHQT1.0 size=158125 2024-03-04T17:50:23.232 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.232+0000 7fe960f568c0 1 < canceled multipart upload key=P28N6U79ZUV2 upload=R6MMON1SHQT1 tag=N43EP4RBDP6L type=write size=1206701 2024-03-04T17:50:23.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.236+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=RL14EUY059C2 type=write size=954235 2024-03-04T17:50:23.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.240+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~L0N2F5RUZAVQ.0 size=1048576 2024-03-04T17:50:23.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.240+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~L0N2F5RUZAVQ.1 size=870668 2024-03-04T17:50:23.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.248+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=L0N2F5RUZAVQ tag=M14UJ3DOMW7P type=write size=2967820 2024-03-04T17:50:23.254 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.252+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~LTT9O82FIL40.0 size=974597 2024-03-04T17:50:23.254 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.252+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=LTT9O82FIL40 tag=LGTAVQKWUOST type=write size=2023173 2024-03-04T17:50:23.260 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.260+0000 7fe960f568c0 1 < suggested remove operation key=MIZRUP4MEA0J tag=4ZPN7UA2YELH type=del 2024-03-04T17:50:23.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.264+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=ITGKVPTKWQDR type=del size=538652 2024-03-04T17:50:23.271 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.272+0000 7fe960f568c0 1 < suggested remove operation key=78N8HHC4O4CG tag=7XS8DCG81PLR type=del 2024-03-04T17:50:23.279 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.276+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~SMAAMOF9L7RU.0 size=1048576 2024-03-04T17:50:23.282 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.280+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~SMAAMOF9L7RU.1 size=789485 2024-03-04T17:50:23.282 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.280+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=SMAAMOF9L7RU tag=Y8TF23L5K005 type=write size=2886637 2024-03-04T17:50:23.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.284+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=RJEE7ORV17VB type=del 2024-03-04T17:50:23.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.288+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=1Y6UE3FWUZGE type=del size=3959886 2024-03-04T17:50:23.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.292+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=M3GE8RREBXGR type=del 2024-03-04T17:50:23.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.296+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=BX2HPK2UXU3P type=write size=971714 2024-03-04T17:50:23.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.304+0000 7fe960f568c0 1 < suggested update operation key=8PZD01OOX3AF tag=0DN1N9X4ZYZV type=del size=1219106 2024-03-04T17:50:23.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.308+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=1NFF555YQIBI type=write size=625990 2024-03-04T17:50:23.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.312+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=2Z0PB7N0X2K6 type=del 2024-03-04T17:50:23.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.316+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~LJBVXCUBAM1U.0 size=1048576 2024-03-04T17:50:23.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.320+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~LJBVXCUBAM1U.1 size=489534 2024-03-04T17:50:23.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.320+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=LJBVXCUBAM1U tag=AK2X7G9PHA5K type=write size=2586686 2024-03-04T17:50:23.326 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.324+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=521RYGA0NBO4 type=del 2024-03-04T17:50:23.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.328+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=ZFJNLZHACXLK type=del size=1211280 2024-03-04T17:50:23.336 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.336+0000 7fe960f568c0 1 < suggested remove operation key=BFF9OD00TU54 tag=QXJ35TFEY786 type=del 2024-03-04T17:50:23.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.340+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=J23IH6EAOUMQ type=del size=1323040 2024-03-04T17:50:23.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.344+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~B3S6FLRJKJWU.0 size=1048576 2024-03-04T17:50:23.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.344+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~B3S6FLRJKJWU.1 size=146204 2024-03-04T17:50:23.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.352+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=B3S6FLRJKJWU tag=17B0PJ9GSRIC type=write size=2243356 2024-03-04T17:50:23.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.356+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=COEOXT0LQY5V type=del size=2443655 2024-03-04T17:50:23.360 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.360+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~DNKQEXUQHW94.0 size=448027 2024-03-04T17:50:23.366 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.364+0000 7fe960f568c0 1 < completed multipart upload key=U2G6YEVD5506 upload=DNKQEXUQHW94 tag=X92EK56RMDTE type=write size=1496603 2024-03-04T17:50:23.370 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.368+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=P1GTTQCSM0K9 type=del size=4046501 2024-03-04T17:50:23.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.372+0000 7fe960f568c0 1 < canceled operation key=GZ4RRIR5CC26 tag=T0XE6SCPMXU2 type=del size=4051900 2024-03-04T17:50:23.377 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.376+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=6JBU3QJSTW3O type=write size=736262 2024-03-04T17:50:23.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.380+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~DVM88MNWYC1U.0 size=1048576 2024-03-04T17:50:23.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.384+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~DVM88MNWYC1U.1 size=405252 2024-03-04T17:50:23.391 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.388+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=DVM88MNWYC1U tag=QSIIKQ2ETL35 type=write size=2502404 2024-03-04T17:50:23.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.396+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=NNO6Z01NG46U type=del size=2650860 2024-03-04T17:50:23.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.400+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=S0PONH9D3J9Y type=del 2024-03-04T17:50:23.404 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.404+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=4YRAD01B2CE7 type=del size=3084904 2024-03-04T17:50:23.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.408+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=4VVI6N3CN1U7 type=del 2024-03-04T17:50:23.415 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.412+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~L690K5PL11DQ.0 size=1048576 2024-03-04T17:50:23.417 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.416+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~L690K5PL11DQ.1 size=823632 2024-03-04T17:50:23.418 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.416+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=L690K5PL11DQ tag=SLK9UFA6G4VU type=write size=2920784 2024-03-04T17:50:23.421 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.420+0000 7fe960f568c0 1 < canceled operation key=RII14ZGX2EL1 tag=RL14EUY059C2 type=write size=954235 2024-03-04T17:50:23.425 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.424+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=2KCHKAM3KRJ7 type=del size=523681 2024-03-04T17:50:23.428 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.428+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~LTT9O82FIL40.0 size=974597 2024-03-04T17:50:23.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.432+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=LTT9O82FIL40 tag=LGTAVQKWUOST type=write size=2023173 2024-03-04T17:50:23.441 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.440+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~GTJTXLKEG46E.0 size=1048576 2024-03-04T17:50:23.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.444+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~GTJTXLKEG46E.1 size=1048576 2024-03-04T17:50:23.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.448+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~GTJTXLKEG46E.2 size=413255 2024-03-04T17:50:23.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.448+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=GTJTXLKEG46E tag=B3B7P2ZO47E9 type=write size=3558983 2024-03-04T17:50:23.454 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.452+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=ITGKVPTKWQDR type=del 2024-03-04T17:50:23.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.456+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=VYY8J5JJ1V58 type=del size=1215121 2024-03-04T17:50:23.462 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.460+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~SMAAMOF9L7RU.0 size=1048576 2024-03-04T17:50:23.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.464+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~SMAAMOF9L7RU.1 size=789485 2024-03-04T17:50:23.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.468+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=SMAAMOF9L7RU tag=Y8TF23L5K005 type=write size=2886637 2024-03-04T17:50:23.478 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.476+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~L8VBFHA7OQ3V.0 size=1048576 2024-03-04T17:50:23.480 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.480+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~L8VBFHA7OQ3V.1 size=1048576 2024-03-04T17:50:23.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.484+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~L8VBFHA7OQ3V.2 size=632540 2024-03-04T17:50:23.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.484+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=L8VBFHA7OQ3V tag=JN2PDH4ZRSCU type=write size=3778268 2024-03-04T17:50:23.487 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.484+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=1Y6UE3FWUZGE type=del 2024-03-04T17:50:23.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.488+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=UD72HYVT76UT type=del size=357757 2024-03-04T17:50:23.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.492+0000 7fe960f568c0 1 < completed write operation key=ASC735ZCBLCG tag=BX2HPK2UXU3P type=write size=971714 2024-03-04T17:50:23.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.500+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~KZN585E69DBK.0 size=1048576 2024-03-04T17:50:23.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.504+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~KZN585E69DBK.1 size=9277 2024-03-04T17:50:23.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.504+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=KZN585E69DBK tag=RV7N4ZM1L4SP type=write size=2106429 2024-03-04T17:50:23.513 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.512+0000 7fe960f568c0 1 < suggested update operation key=QGLN27WH1N2O tag=1NFF555YQIBI type=write size=625990 2024-03-04T17:50:23.516 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.516+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=4WCM9P6V2RI9 type=del size=1240186 2024-03-04T17:50:23.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.520+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~LJBVXCUBAM1U.0 size=1048576 2024-03-04T17:50:23.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.520+0000 7fe960f568c0 1 < canceled part key=_multipart_2FUTMG0SSXBY.2~LJBVXCUBAM1U.1 size=489534 2024-03-04T17:50:23.527 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.528+0000 7fe960f568c0 1 < canceled multipart upload key=2FUTMG0SSXBY upload=LJBVXCUBAM1U tag=AK2X7G9PHA5K type=write size=2586686 2024-03-04T17:50:23.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.532+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~3O65ADFWERY0.0 size=1048576 2024-03-04T17:50:23.537 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.536+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~3O65ADFWERY0.1 size=1048576 2024-03-04T17:50:23.540 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.540+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~3O65ADFWERY0.2 size=730166 2024-03-04T17:50:23.540 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.540+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=3O65ADFWERY0 tag=6E7GYPQ4W4QV type=write size=3875894 2024-03-04T17:50:23.544 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.544+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=ZFJNLZHACXLK type=del 2024-03-04T17:50:23.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.548+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~K68H0E8PFQUO.0 size=1048576 2024-03-04T17:50:23.553 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.552+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~K68H0E8PFQUO.1 size=833623 2024-03-04T17:50:23.553 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.552+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=K68H0E8PFQUO tag=FSJ5O5I4O7RB type=write size=2930775 2024-03-04T17:50:23.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.556+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=J23IH6EAOUMQ type=del 2024-03-04T17:50:23.561 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.560+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=NW37A2N7P08D type=del size=3781243 2024-03-04T17:50:23.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.568+0000 7fe960f568c0 1 < suggested remove operation key=2FUTMG0SSXBY tag=COEOXT0LQY5V type=del 2024-03-04T17:50:23.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.572+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=56ZCVA9GXH34 type=del size=2872323 2024-03-04T17:50:23.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.580+0000 7fe960f568c0 1 < suggested remove operation key=RII14ZGX2EL1 tag=P1GTTQCSM0K9 type=del 2024-03-04T17:50:23.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.584+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=KWD7050A1B41 type=del size=872903 2024-03-04T17:50:23.590 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.588+0000 7fe960f568c0 1 < completed write operation key=WNXVI6K5MU3R tag=6JBU3QJSTW3O type=write size=736262 2024-03-04T17:50:23.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.596+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~PJHNWPIXLM1N.0 size=1048576 2024-03-04T17:50:23.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.596+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~PJHNWPIXLM1N.1 size=1048576 2024-03-04T17:50:23.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.600+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~PJHNWPIXLM1N.2 size=939627 2024-03-04T17:50:23.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.600+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=PJHNWPIXLM1N tag=OEKQMA8259TZ type=write size=4085355 2024-03-04T17:50:23.605 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.604+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=NNO6Z01NG46U type=del 2024-03-04T17:50:23.609 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.608+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=SCR429W16BL0 type=del size=2378381 2024-03-04T17:50:23.613 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.612+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=4YRAD01B2CE7 type=del 2024-03-04T17:50:23.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.620+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~IIZ1WLR2ASBB.0 size=1048576 2024-03-04T17:50:23.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.620+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~IIZ1WLR2ASBB.1 size=902302 2024-03-04T17:50:23.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.620+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=IIZ1WLR2ASBB tag=67DKP9EWA0RU type=write size=2999454 2024-03-04T17:50:23.626 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.624+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~L690K5PL11DQ.0 size=1048576 2024-03-04T17:50:23.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.628+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~L690K5PL11DQ.1 size=823632 2024-03-04T17:50:23.635 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.636+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=L690K5PL11DQ tag=SLK9UFA6G4VU type=write size=2920784 2024-03-04T17:50:23.639 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.640+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=GUGOVWQ9VETF type=del size=4015310 2024-03-04T17:50:23.643 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.644+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=2KCHKAM3KRJ7 type=del 2024-03-04T17:50:23.647 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.648+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=E4VLQYGIJEL5 type=del size=1050687 2024-03-04T17:50:23.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.652+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~GTJTXLKEG46E.0 size=1048576 2024-03-04T17:50:23.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.652+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~GTJTXLKEG46E.1 size=1048576 2024-03-04T17:50:23.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.656+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~GTJTXLKEG46E.2 size=413255 2024-03-04T17:50:23.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.664+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=GTJTXLKEG46E tag=B3B7P2ZO47E9 type=write size=3558983 2024-03-04T17:50:23.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.668+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=6E1C7U1YO7GZ type=del size=2484898 2024-03-04T17:50:23.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.672+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=VYY8J5JJ1V58 type=del 2024-03-04T17:50:23.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.676+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=0YF19J5BVD9T type=del size=2543951 2024-03-04T17:50:23.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.684+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~L8VBFHA7OQ3V.0 size=1048576 2024-03-04T17:50:23.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.684+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~L8VBFHA7OQ3V.1 size=1048576 2024-03-04T17:50:23.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.688+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~L8VBFHA7OQ3V.2 size=632540 2024-03-04T17:50:23.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.696+0000 7fe960f568c0 1 < completed multipart upload key=2MKQA5JYWU8L upload=L8VBFHA7OQ3V tag=JN2PDH4ZRSCU type=write size=3778268 2024-03-04T17:50:23.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.704+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~04CJG7VUQLLM.0 size=1048576 2024-03-04T17:50:23.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.708+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~04CJG7VUQLLM.1 size=34810 2024-03-04T17:50:23.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.708+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=04CJG7VUQLLM tag=L8GIL2RO8XJ3 type=write size=2131962 2024-03-04T17:50:23.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.716+0000 7fe960f568c0 1 < suggested remove operation key=NF4OIJWJ83DE tag=UD72HYVT76UT type=del 2024-03-04T17:50:23.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.720+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~30H13XNOSUK3.0 size=1048576 2024-03-04T17:50:23.725 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.724+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~30H13XNOSUK3.1 size=1048576 2024-03-04T17:50:23.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.728+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~30H13XNOSUK3.2 size=216867 2024-03-04T17:50:23.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.728+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=30H13XNOSUK3 tag=6KTAJ8VYIZTU type=write size=3362595 2024-03-04T17:50:23.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.732+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~KZN585E69DBK.0 size=1048576 2024-03-04T17:50:23.735 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.732+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~KZN585E69DBK.1 size=9277 2024-03-04T17:50:23.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.740+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=KZN585E69DBK tag=RV7N4ZM1L4SP type=write size=2106429 2024-03-04T17:50:23.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.744+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=BCPEJFEEKBOA type=write size=588640 2024-03-04T17:50:23.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.752+0000 7fe960f568c0 1 < suggested update operation key=UKZ079SVZZKG tag=4WCM9P6V2RI9 type=del size=1240186 2024-03-04T17:50:23.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.756+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=7UHTTR2CL9S8 type=del size=3079466 2024-03-04T17:50:23.760 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.760+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~3O65ADFWERY0.0 size=1048576 2024-03-04T17:50:23.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.764+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~3O65ADFWERY0.1 size=1048576 2024-03-04T17:50:23.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.764+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~3O65ADFWERY0.2 size=730166 2024-03-04T17:50:23.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.772+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=3O65ADFWERY0 tag=6E7GYPQ4W4QV type=write size=3875894 2024-03-04T17:50:23.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.780+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~UFJPZ46CJLJG.0 size=1048576 2024-03-04T17:50:23.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.780+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~UFJPZ46CJLJG.1 size=1015293 2024-03-04T17:50:23.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.780+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=UFJPZ46CJLJG tag=4H082GTDIVRC type=write size=3112445 2024-03-04T17:50:23.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.784+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~K68H0E8PFQUO.0 size=1048576 2024-03-04T17:50:23.789 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.788+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~K68H0E8PFQUO.1 size=833623 2024-03-04T17:50:23.795 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.796+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=K68H0E8PFQUO tag=FSJ5O5I4O7RB type=write size=2930775 2024-03-04T17:50:23.803 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.804+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=554KW9G2IF5Z type=del size=267303 2024-03-04T17:50:23.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.804+0000 7fe960f568c0 -1 < failed to complete operation key=RII14ZGX2EL1 tag=NW37A2N7P08D type=del size=3781243: (22) Invalid argument 2024-03-04T17:50:23.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.812+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~TP66TFLOCI2J.0 size=1048576 2024-03-04T17:50:23.816 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.816+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~TP66TFLOCI2J.1 size=1048576 2024-03-04T17:50:23.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.816+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~TP66TFLOCI2J.2 size=165316 2024-03-04T17:50:23.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.816+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=TP66TFLOCI2J tag=Z2U0BHMTC13S type=write size=3311044 2024-03-04T17:50:23.822 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.820+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=56ZCVA9GXH34 type=del 2024-03-04T17:50:23.826 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.824+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=1NJH7Z0OGC8T type=del size=1548385 2024-03-04T17:50:23.830 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.828+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=KWD7050A1B41 type=del 2024-03-04T17:50:23.834 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.832+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=KFY4STT07O0K type=del size=4128556 2024-03-04T17:50:23.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.836+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~PJHNWPIXLM1N.0 size=1048576 2024-03-04T17:50:23.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.840+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~PJHNWPIXLM1N.1 size=1048576 2024-03-04T17:50:23.843 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.840+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~PJHNWPIXLM1N.2 size=939627 2024-03-04T17:50:23.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.848+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=PJHNWPIXLM1N tag=OEKQMA8259TZ type=write size=4085355 2024-03-04T17:50:23.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.856+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~6U7LC90UPX85.0 size=1048576 2024-03-04T17:50:23.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.856+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~6U7LC90UPX85.1 size=1048576 2024-03-04T17:50:23.861 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.860+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~6U7LC90UPX85.2 size=385857 2024-03-04T17:50:23.861 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.860+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=6U7LC90UPX85 tag=OSSX308QQP10 type=write size=3531585 2024-03-04T17:50:23.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.864+0000 7fe960f568c0 1 < completed delete operation key=ASC735ZCBLCG tag=SCR429W16BL0 type=del 2024-03-04T17:50:23.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.868+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=829M0264NZ5Q type=del size=1572800 2024-03-04T17:50:23.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.868+0000 7fe960f568c0 1 < canceled part key=_multipart_U2G6YEVD5506.2~IIZ1WLR2ASBB.0 size=1048576 2024-03-04T17:50:23.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.872+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~IIZ1WLR2ASBB.1 size=902302 2024-03-04T17:50:23.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.876+0000 7fe960f568c0 1 < canceled multipart upload key=U2G6YEVD5506 upload=IIZ1WLR2ASBB tag=67DKP9EWA0RU type=write size=2999454 2024-03-04T17:50:23.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.880+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=WR3EF79VNQ4W type=del size=3512189 2024-03-04T17:50:23.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.888+0000 7fe960f568c0 1 < suggested remove operation key=RII14ZGX2EL1 tag=GUGOVWQ9VETF type=del 2024-03-04T17:50:23.893 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.892+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=O5O25ANI4300 type=del size=1132293 2024-03-04T17:50:23.896 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.896+0000 7fe960f568c0 -1 < failed to complete operation key=RII14ZGX2EL1 tag=E4VLQYGIJEL5 type=del size=1050687: (22) Invalid argument 2024-03-04T17:50:23.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.904+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~THWM50QH17PP.0 size=711229 2024-03-04T17:50:23.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.904+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=THWM50QH17PP tag=6TKP0C0YCY3O type=write size=1759805 2024-03-04T17:50:23.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.904+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=6E1C7U1YO7GZ type=del 2024-03-04T17:50:23.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.908+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=HPP2A136K12T type=del size=5719 2024-03-04T17:50:23.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.912+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=0YF19J5BVD9T type=del 2024-03-04T17:50:23.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.920+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~89Q92T3T3231.0 size=1048576 2024-03-04T17:50:23.924 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.924+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~89Q92T3T3231.1 size=1037242 2024-03-04T17:50:23.925 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.924+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=89Q92T3T3231 tag=AMY09SS69OD3 type=write size=3134394 2024-03-04T17:50:23.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.928+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~04CJG7VUQLLM.0 size=1048576 2024-03-04T17:50:23.933 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.932+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~04CJG7VUQLLM.1 size=34810 2024-03-04T17:50:23.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.940+0000 7fe960f568c0 1 < completed multipart upload key=XSQ3GE2UHMZG upload=04CJG7VUQLLM tag=L8GIL2RO8XJ3 type=write size=2131962 2024-03-04T17:50:23.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.944+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=RNEPMDA845S4 type=del size=741082 2024-03-04T17:50:23.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.948+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~30H13XNOSUK3.0 size=1048576 2024-03-04T17:50:23.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.952+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~30H13XNOSUK3.1 size=1048576 2024-03-04T17:50:23.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.956+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~30H13XNOSUK3.2 size=216867 2024-03-04T17:50:23.962 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.960+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=30H13XNOSUK3 tag=6KTAJ8VYIZTU type=write size=3362595 2024-03-04T17:50:23.966 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.964+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=C52WE0B43RNS type=del size=217844 2024-03-04T17:50:23.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.972+0000 7fe960f568c0 1 < suggested remove operation key=2MKQA5JYWU8L tag=BCPEJFEEKBOA type=write 2024-03-04T17:50:23.978 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.976+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=ROQK62QN1Q7L type=del size=643108 2024-03-04T17:50:23.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.980+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=7UHTTR2CL9S8 type=del 2024-03-04T17:50:23.989 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.988+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~B6VS3GV6UMZA.0 size=1048576 2024-03-04T17:50:23.991 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.992+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~B6VS3GV6UMZA.1 size=4313 2024-03-04T17:50:23.992 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.992+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=B6VS3GV6UMZA tag=17WX4VAGLQKO type=write size=2101465 2024-03-04T17:50:23.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.996+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~UFJPZ46CJLJG.0 size=1048576 2024-03-04T17:50:23.999 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:23.996+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~UFJPZ46CJLJG.1 size=1015293 2024-03-04T17:50:24.004 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.004+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=UFJPZ46CJLJG tag=4H082GTDIVRC type=write size=3112445 2024-03-04T17:50:24.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.008+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=TFFIMDVLD5GH type=del size=3316018 2024-03-04T17:50:24.012 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.012+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=554KW9G2IF5Z type=del 2024-03-04T17:50:24.019 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.016+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~IRD9XWQKC184.0 size=1048576 2024-03-04T17:50:24.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.020+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~IRD9XWQKC184.1 size=599371 2024-03-04T17:50:24.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.020+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=IRD9XWQKC184 tag=2S5IFMSBU7WX type=write size=2696523 2024-03-04T17:50:24.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.024+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~TP66TFLOCI2J.0 size=1048576 2024-03-04T17:50:24.028 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.028+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~TP66TFLOCI2J.1 size=1048576 2024-03-04T17:50:24.031 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.028+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~TP66TFLOCI2J.2 size=165316 2024-03-04T17:50:24.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.036+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=TP66TFLOCI2J tag=Z2U0BHMTC13S type=write size=3311044 2024-03-04T17:50:24.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.040+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=62IP3WQDEC2P type=write size=286480 2024-03-04T17:50:24.044 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.044+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=1NJH7Z0OGC8T type=del 2024-03-04T17:50:24.051 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.048+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~UGKGX229B05E.0 size=1048576 2024-03-04T17:50:24.054 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.052+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~UGKGX229B05E.1 size=1048576 2024-03-04T17:50:24.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.056+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~UGKGX229B05E.2 size=665046 2024-03-04T17:50:24.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.056+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=UGKGX229B05E tag=7660WPSVOITU type=write size=3810774 2024-03-04T17:50:24.060 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.060+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=KFY4STT07O0K type=del 2024-03-04T17:50:24.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.064+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=L6KXA59TZ4NZ type=del size=1842563 2024-03-04T17:50:24.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.068+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~6U7LC90UPX85.0 size=1048576 2024-03-04T17:50:24.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.072+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~6U7LC90UPX85.1 size=1048576 2024-03-04T17:50:24.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.072+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~6U7LC90UPX85.2 size=385857 2024-03-04T17:50:24.080 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.080+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=6U7LC90UPX85 tag=OSSX308QQP10 type=write size=3531585 2024-03-04T17:50:24.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.084+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=F09PVM79XWVK type=del size=1695789 2024-03-04T17:50:24.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.088+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=829M0264NZ5Q type=del 2024-03-04T17:50:24.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.092+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~IBX10QN6P6GQ.0 size=1048576 2024-03-04T17:50:24.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.096+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~IBX10QN6P6GQ.1 size=1048576 2024-03-04T17:50:24.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.100+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~IBX10QN6P6GQ.2 size=483305 2024-03-04T17:50:24.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.100+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=IBX10QN6P6GQ tag=C4D87AVPHECT type=write size=3629033 2024-03-04T17:50:24.103 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.100+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=WR3EF79VNQ4W type=del 2024-03-04T17:50:24.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.104+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=92X4WECKY0PI type=del size=2262588 2024-03-04T17:50:24.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.108+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=O5O25ANI4300 type=del 2024-03-04T17:50:24.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.112+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=9VDKSWCMI7SO type=del size=323675 2024-03-04T17:50:24.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.116+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~THWM50QH17PP.0 size=711229 2024-03-04T17:50:24.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.124+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=THWM50QH17PP tag=6TKP0C0YCY3O type=write size=1759805 2024-03-04T17:50:24.128 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.128+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=Q0F9JJG7FOY0 type=del size=3664929 2024-03-04T17:50:24.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.132+0000 7fe960f568c0 1 < canceled operation key=BFF9OD00TU54 tag=HPP2A136K12T type=del size=5719 2024-03-04T17:50:24.136 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.136+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=ZLWJZ1HRJ24D type=del size=87858 2024-03-04T17:50:24.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.140+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~89Q92T3T3231.0 size=1048576 2024-03-04T17:50:24.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.140+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~89Q92T3T3231.1 size=1037242 2024-03-04T17:50:24.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.152+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=89Q92T3T3231 tag=AMY09SS69OD3 type=write size=3134394 2024-03-04T17:50:24.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.156+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~TPQCGQ4DVCG9.0 size=1048576 2024-03-04T17:50:24.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.160+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~TPQCGQ4DVCG9.1 size=1048576 2024-03-04T17:50:24.163 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.160+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~TPQCGQ4DVCG9.2 size=816830 2024-03-04T17:50:24.163 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.160+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=TPQCGQ4DVCG9 tag=UZQJU8L7SWJY type=write size=3962558 2024-03-04T17:50:24.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.164+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=RNEPMDA845S4 type=del 2024-03-04T17:50:24.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.172+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~7M2CYHANN754.0 size=1048576 2024-03-04T17:50:24.174 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.172+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~7M2CYHANN754.1 size=1048576 2024-03-04T17:50:24.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.176+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~7M2CYHANN754.2 size=1025535 2024-03-04T17:50:24.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.176+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=7M2CYHANN754 tag=NC41YDOBHM2L type=write size=4171263 2024-03-04T17:50:24.180 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.180+0000 7fe960f568c0 1 < canceled operation key=MIZRUP4MEA0J tag=C52WE0B43RNS type=del size=217844 2024-03-04T17:50:24.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.184+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~KRW6YUBLI91D.0 size=403690 2024-03-04T17:50:24.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.184+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=KRW6YUBLI91D tag=NMDNCZ3NBSZO type=write size=1452266 2024-03-04T17:50:24.189 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.188+0000 7fe960f568c0 1 < canceled operation key=BFF9OD00TU54 tag=ROQK62QN1Q7L type=del size=643108 2024-03-04T17:50:24.192 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.192+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=IPZTW67R4ICI type=del size=3742038 2024-03-04T17:50:24.196 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.196+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~B6VS3GV6UMZA.0 size=1048576 2024-03-04T17:50:24.199 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.196+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~B6VS3GV6UMZA.1 size=4313 2024-03-04T17:50:24.205 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.204+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=B6VS3GV6UMZA tag=17WX4VAGLQKO type=write size=2101465 2024-03-04T17:50:24.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.208+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=B79G2Q8824MO type=write size=172723 2024-03-04T17:50:24.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.212+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=TFFIMDVLD5GH type=del 2024-03-04T17:50:24.219 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.216+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~FV3NDR5IO47J.0 size=1048576 2024-03-04T17:50:24.221 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.220+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~FV3NDR5IO47J.1 size=1002650 2024-03-04T17:50:24.221 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.220+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=FV3NDR5IO47J tag=J1O5YDRGHZ97 type=write size=3099802 2024-03-04T17:50:24.225 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.224+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~IRD9XWQKC184.0 size=1048576 2024-03-04T17:50:24.227 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.228+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~IRD9XWQKC184.1 size=599371 2024-03-04T17:50:24.233 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.232+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=IRD9XWQKC184 tag=2S5IFMSBU7WX type=write size=2696523 2024-03-04T17:50:24.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.236+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~8JGI4H624WEL.0 size=1031820 2024-03-04T17:50:24.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.236+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=8JGI4H624WEL tag=8BMIPE1DFGOW type=write size=2080396 2024-03-04T17:50:24.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.240+0000 7fe960f568c0 1 < canceled operation key=MIZRUP4MEA0J tag=62IP3WQDEC2P type=write size=286480 2024-03-04T17:50:24.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.248+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~BSK5ND8HE3Q9.0 size=1048576 2024-03-04T17:50:24.252 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.252+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~BSK5ND8HE3Q9.1 size=1048576 2024-03-04T17:50:24.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.252+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~BSK5ND8HE3Q9.2 size=536695 2024-03-04T17:50:24.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.252+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=BSK5ND8HE3Q9 tag=WEW78XWF2PMY type=write size=3682423 2024-03-04T17:50:24.256 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.256+0000 7fe960f568c0 1 < canceled part key=_multipart_NF4OIJWJ83DE.2~UGKGX229B05E.0 size=1048576 2024-03-04T17:50:24.259 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.260+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~UGKGX229B05E.1 size=1048576 2024-03-04T17:50:24.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.260+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~UGKGX229B05E.2 size=665046 2024-03-04T17:50:24.269 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.268+0000 7fe960f568c0 1 < canceled multipart upload key=NF4OIJWJ83DE upload=UGKGX229B05E tag=7660WPSVOITU type=write size=3810774 2024-03-04T17:50:24.277 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.276+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~F8GIRGDWXM3R.0 size=1048576 2024-03-04T17:50:24.280 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.280+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~F8GIRGDWXM3R.1 size=949770 2024-03-04T17:50:24.280 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.280+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=F8GIRGDWXM3R tag=FYOTE49JR23G type=write size=3046922 2024-03-04T17:50:24.285 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.284+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=L6KXA59TZ4NZ type=del 2024-03-04T17:50:24.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.288+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=BMOV6CLTNK8J type=write size=534467 2024-03-04T17:50:24.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.292+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=F09PVM79XWVK type=del 2024-03-04T17:50:24.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.296+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=4TQK21PJSME1 type=write size=555293 2024-03-04T17:50:24.300 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.300+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~IBX10QN6P6GQ.0 size=1048576 2024-03-04T17:50:24.300 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.300+0000 7fe960f568c0 1 < canceled part key=_multipart_P28N6U79ZUV2.2~IBX10QN6P6GQ.1 size=1048576 2024-03-04T17:50:24.303 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.304+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~IBX10QN6P6GQ.2 size=483305 2024-03-04T17:50:24.308 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.308+0000 7fe960f568c0 1 < canceled multipart upload key=P28N6U79ZUV2 upload=IBX10QN6P6GQ tag=C4D87AVPHECT type=write size=3629033 2024-03-04T17:50:24.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.312+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=588MRLFY1D02 type=del size=1028569 2024-03-04T17:50:24.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.316+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=92X4WECKY0PI type=del 2024-03-04T17:50:24.320 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.320+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=ICABRMPDTE9I type=write size=198524 2024-03-04T17:50:24.324 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.324+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=9VDKSWCMI7SO type=del 2024-03-04T17:50:24.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.328+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=YYUZDJRVE7MZ type=del size=1375578 2024-03-04T17:50:24.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.332+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=Q0F9JJG7FOY0 type=del 2024-03-04T17:50:24.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.336+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~E7IR604HCABK.0 size=1048576 2024-03-04T17:50:24.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.340+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~E7IR604HCABK.1 size=1048576 2024-03-04T17:50:24.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.344+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~E7IR604HCABK.2 size=875743 2024-03-04T17:50:24.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.344+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=E7IR604HCABK tag=GDPU4TACM1DF type=write size=4021471 2024-03-04T17:50:24.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.348+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=ZLWJZ1HRJ24D type=del 2024-03-04T17:50:24.351 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.348+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=8T84G82LWM1I type=del size=3486534 2024-03-04T17:50:24.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.352+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~TPQCGQ4DVCG9.0 size=1048576 2024-03-04T17:50:24.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.356+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~TPQCGQ4DVCG9.1 size=1048576 2024-03-04T17:50:24.360 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.360+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~TPQCGQ4DVCG9.2 size=816830 2024-03-04T17:50:24.366 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.364+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=TPQCGQ4DVCG9 tag=UZQJU8L7SWJY type=write size=3962558 2024-03-04T17:50:24.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.368+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=70YPF3ZX5SCM type=write size=989014 2024-03-04T17:50:24.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.376+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~7M2CYHANN754.0 size=1048576 2024-03-04T17:50:24.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.380+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~7M2CYHANN754.1 size=1048576 2024-03-04T17:50:24.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.380+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~7M2CYHANN754.2 size=1025535 2024-03-04T17:50:24.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.388+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=7M2CYHANN754 tag=NC41YDOBHM2L type=write size=4171263 2024-03-04T17:50:24.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.392+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~RX8CDXERY062.0 size=560367 2024-03-04T17:50:24.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.392+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=RX8CDXERY062 tag=SE1QZL04MLHW type=write size=1608943 2024-03-04T17:50:24.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.396+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~KRW6YUBLI91D.0 size=403690 2024-03-04T17:50:24.404 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.404+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=KRW6YUBLI91D tag=NMDNCZ3NBSZO type=write size=1452266 2024-03-04T17:50:24.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.408+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=22SJ3J8AULZ8 type=del size=1260898 2024-03-04T17:50:24.412 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.412+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=IPZTW67R4ICI type=del 2024-03-04T17:50:24.415 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.416+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=M3JI8HQYF50V type=del size=4090860 2024-03-04T17:50:24.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.420+0000 7fe960f568c0 1 < completed write operation key=GZ4RRIR5CC26 tag=B79G2Q8824MO type=write size=172723 2024-03-04T17:50:24.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.424+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~AEMFG8RCQLBC.0 size=1048576 2024-03-04T17:50:24.430 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.428+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~AEMFG8RCQLBC.1 size=442268 2024-03-04T17:50:24.430 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.428+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=AEMFG8RCQLBC tag=YNNJT5OXEU07 type=write size=2539420 2024-03-04T17:50:24.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.432+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~FV3NDR5IO47J.0 size=1048576 2024-03-04T17:50:24.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.436+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~FV3NDR5IO47J.1 size=1002650 2024-03-04T17:50:24.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.444+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=FV3NDR5IO47J tag=J1O5YDRGHZ97 type=write size=3099802 2024-03-04T17:50:24.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.448+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=UMBB2VF4PDK3 type=del size=3335552 2024-03-04T17:50:24.454 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.452+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~8JGI4H624WEL.0 size=1031820 2024-03-04T17:50:24.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.460+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=8JGI4H624WEL tag=8BMIPE1DFGOW type=write size=2080396 2024-03-04T17:50:24.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.464+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=OUZDYPMFLV2F type=del size=828787 2024-03-04T17:50:24.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.468+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~BSK5ND8HE3Q9.0 size=1048576 2024-03-04T17:50:24.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.472+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~BSK5ND8HE3Q9.1 size=1048576 2024-03-04T17:50:24.476 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.476+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~BSK5ND8HE3Q9.2 size=536695 2024-03-04T17:50:24.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.480+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=BSK5ND8HE3Q9 tag=WEW78XWF2PMY type=write size=3682423 2024-03-04T17:50:24.494 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.492+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~B8XCP0JBVHWD.0 size=1048576 2024-03-04T17:50:24.497 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.496+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~B8XCP0JBVHWD.1 size=1048576 2024-03-04T17:50:24.500 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.500+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~B8XCP0JBVHWD.2 size=133320 2024-03-04T17:50:24.500 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.500+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=B8XCP0JBVHWD tag=3YQIUDCM03EH type=write size=3279048 2024-03-04T17:50:24.503 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.504+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~F8GIRGDWXM3R.0 size=1048576 2024-03-04T17:50:24.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.504+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~F8GIRGDWXM3R.1 size=949770 2024-03-04T17:50:24.512 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.512+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=F8GIRGDWXM3R tag=FYOTE49JR23G type=write size=3046922 2024-03-04T17:50:24.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.520+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~WPK2YF3VVN4O.0 size=1048576 2024-03-04T17:50:24.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.520+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~WPK2YF3VVN4O.1 size=1048576 2024-03-04T17:50:24.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.524+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~WPK2YF3VVN4O.2 size=1031624 2024-03-04T17:50:24.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.524+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=WPK2YF3VVN4O tag=U563UP441ZRW type=write size=4177352 2024-03-04T17:50:24.529 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.528+0000 7fe960f568c0 1 < completed write operation key=3YTAYUQLLUKY tag=BMOV6CLTNK8J type=write size=534467 2024-03-04T17:50:24.535 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.536+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~FPJ8Q1G1I846.0 size=331904 2024-03-04T17:50:24.535 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.536+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=FPJ8Q1G1I846 tag=HBG3XCE5M2GE type=write size=1380480 2024-03-04T17:50:24.539 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.536+0000 7fe960f568c0 1 < completed write operation key=QGLN27WH1N2O tag=4TQK21PJSME1 type=write size=555293 2024-03-04T17:50:24.544 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.544+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=7JGJ5RVGFGNS type=del size=1663078 2024-03-04T17:50:24.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.552+0000 7fe960f568c0 1 < suggested update operation key=BFF9OD00TU54 tag=588MRLFY1D02 type=del size=1028569 2024-03-04T17:50:24.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.552+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=DIY5QRUYTPNG type=del size=492195 2024-03-04T17:50:24.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.556+0000 7fe960f568c0 1 < completed write operation key=RII14ZGX2EL1 tag=ICABRMPDTE9I type=write size=198524 2024-03-04T17:50:24.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.564+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~TMH0HTX7LHNA.0 size=598637 2024-03-04T17:50:24.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.564+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=TMH0HTX7LHNA tag=6L5N6RKCR7RQ type=write size=1647213 2024-03-04T17:50:24.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.568+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=YYUZDJRVE7MZ type=del 2024-03-04T17:50:24.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.572+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~INQH89F8HQF3.0 size=1048576 2024-03-04T17:50:24.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.576+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~INQH89F8HQF3.1 size=946762 2024-03-04T17:50:24.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.576+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=INQH89F8HQF3 tag=EALFKN0TZHCY type=write size=3043914 2024-03-04T17:50:24.581 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.580+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~E7IR604HCABK.0 size=1048576 2024-03-04T17:50:24.581 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.580+0000 7fe960f568c0 1 < canceled part key=_multipart_3X4719GHK3BI.2~E7IR604HCABK.1 size=1048576 2024-03-04T17:50:24.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.584+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~E7IR604HCABK.2 size=875743 2024-03-04T17:50:24.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.588+0000 7fe960f568c0 1 < canceled multipart upload key=3X4719GHK3BI upload=E7IR604HCABK tag=GDPU4TACM1DF type=write size=4021471 2024-03-04T17:50:24.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.592+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=6XS336HDTR3G type=del size=3886900 2024-03-04T17:50:24.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.596+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=8T84G82LWM1I type=del 2024-03-04T17:50:24.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.604+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=YZDDRR57P6JY type=write size=159365 2024-03-04T17:50:24.607 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.608+0000 7fe960f568c0 1 < completed write operation key=JT02WL9JYTNM tag=70YPF3ZX5SCM type=write size=989014 2024-03-04T17:50:24.611 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.608+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=AA4WZFQSBWBY type=write size=842196 2024-03-04T17:50:24.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.616+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~RX8CDXERY062.0 size=560367 2024-03-04T17:50:24.621 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.620+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=RX8CDXERY062 tag=SE1QZL04MLHW type=write size=1608943 2024-03-04T17:50:24.626 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.624+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=E6Y7ZLO44JZP type=write size=474276 2024-03-04T17:50:24.630 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.628+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=22SJ3J8AULZ8 type=del 2024-03-04T17:50:24.633 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.632+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=B60TBFHXVP6B type=del size=3129399 2024-03-04T17:50:24.637 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.636+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=M3JI8HQYF50V type=del 2024-03-04T17:50:24.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.640+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=00XOFYNTTLZO type=write size=192785 2024-03-04T17:50:24.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.644+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~AEMFG8RCQLBC.0 size=1048576 2024-03-04T17:50:24.648 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.648+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~AEMFG8RCQLBC.1 size=442268 2024-03-04T17:50:24.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.652+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=AEMFG8RCQLBC tag=YNNJT5OXEU07 type=write size=2539420 2024-03-04T17:50:24.661 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.660+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~GGBTV3S02UY0.0 size=1048576 2024-03-04T17:50:24.664 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.664+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~GGBTV3S02UY0.1 size=1048576 2024-03-04T17:50:24.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.668+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~GGBTV3S02UY0.2 size=82542 2024-03-04T17:50:24.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.668+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=GGBTV3S02UY0 tag=F32KV646I9RS type=write size=3228270 2024-03-04T17:50:24.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.668+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=UMBB2VF4PDK3 type=del 2024-03-04T17:50:24.675 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.672+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=KUNAEQ0R4S2U type=write size=159392 2024-03-04T17:50:24.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.676+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=OUZDYPMFLV2F type=del 2024-03-04T17:50:24.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.684+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~89END4VAJ5LI.0 size=1048576 2024-03-04T17:50:24.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.688+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~89END4VAJ5LI.1 size=1048576 2024-03-04T17:50:24.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.688+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~89END4VAJ5LI.2 size=523391 2024-03-04T17:50:24.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.688+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=89END4VAJ5LI tag=0COUJ5PV7VSY type=write size=3669119 2024-03-04T17:50:24.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.692+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~B8XCP0JBVHWD.0 size=1048576 2024-03-04T17:50:24.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.696+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~B8XCP0JBVHWD.1 size=1048576 2024-03-04T17:50:24.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.700+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~B8XCP0JBVHWD.2 size=133320 2024-03-04T17:50:24.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.704+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=B8XCP0JBVHWD tag=3YQIUDCM03EH type=write size=3279048 2024-03-04T17:50:24.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.708+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=OM35EBN23BTT type=del size=2057928 2024-03-04T17:50:24.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.712+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~WPK2YF3VVN4O.0 size=1048576 2024-03-04T17:50:24.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.716+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~WPK2YF3VVN4O.1 size=1048576 2024-03-04T17:50:24.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.716+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~WPK2YF3VVN4O.2 size=1031624 2024-03-04T17:50:24.725 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.724+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=WPK2YF3VVN4O tag=U563UP441ZRW type=write size=4177352 2024-03-04T17:50:24.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.732+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~HVDQP468R6HT.0 size=85461 2024-03-04T17:50:24.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.732+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=HVDQP468R6HT tag=I86AKN3FZSGM type=write size=1134037 2024-03-04T17:50:24.735 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.732+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~FPJ8Q1G1I846.0 size=331904 2024-03-04T17:50:24.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.740+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=FPJ8Q1G1I846 tag=HBG3XCE5M2GE type=write size=1380480 2024-03-04T17:50:24.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.744+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=DDH3T4MBHXN7 type=del size=2224949 2024-03-04T17:50:24.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.752+0000 7fe960f568c0 1 < suggested update operation key=QFF67JIW8Y8E tag=7JGJ5RVGFGNS type=del size=1663078 2024-03-04T17:50:24.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.756+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=GMFB696KJJXY type=write size=211459 2024-03-04T17:50:24.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.760+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=DIY5QRUYTPNG type=del 2024-03-04T17:50:24.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.768+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~V1SNP6EW5273.0 size=807943 2024-03-04T17:50:24.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.768+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=V1SNP6EW5273 tag=F1N9Y2X0R3TO type=write size=1856519 2024-03-04T17:50:24.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.772+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~TMH0HTX7LHNA.0 size=598637 2024-03-04T17:50:24.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.776+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=TMH0HTX7LHNA tag=6L5N6RKCR7RQ type=write size=1647213 2024-03-04T17:50:24.785 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.784+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~C62QCLN5QEWP.0 size=463598 2024-03-04T17:50:24.785 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.784+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=C62QCLN5QEWP tag=6Z9VLUNSQXJY type=write size=1512174 2024-03-04T17:50:24.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.788+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~INQH89F8HQF3.0 size=1048576 2024-03-04T17:50:24.793 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.792+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~INQH89F8HQF3.1 size=946762 2024-03-04T17:50:24.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.800+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=INQH89F8HQF3 tag=EALFKN0TZHCY type=write size=3043914 2024-03-04T17:50:24.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.808+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~RUKTFXAJLX18.0 size=1048576 2024-03-04T17:50:24.810 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.808+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~RUKTFXAJLX18.1 size=56275 2024-03-04T17:50:24.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.808+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=RUKTFXAJLX18 tag=1V1BYMV0UYEP type=write size=2153427 2024-03-04T17:50:24.818 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.816+0000 7fe960f568c0 1 < suggested update operation key=UKZ079SVZZKG tag=6XS336HDTR3G type=del size=3886900 2024-03-04T17:50:24.822 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.820+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=GCSUPTP2I5B2 type=del size=2397938 2024-03-04T17:50:24.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.824+0000 7fe960f568c0 -1 < failed to complete operation key=QFF67JIW8Y8E tag=YZDDRR57P6JY type=write size=159365: (22) Invalid argument 2024-03-04T17:50:24.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.832+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~N34RW0DWURTR.0 size=547553 2024-03-04T17:50:24.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.832+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=N34RW0DWURTR tag=WLFLKAGJMDAW type=write size=1596129 2024-03-04T17:50:24.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.836+0000 7fe960f568c0 1 < completed write operation key=2KZ4IFW54C5B tag=AA4WZFQSBWBY type=write size=842196 2024-03-04T17:50:24.843 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.844+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=E3YPK6LLILRI type=del size=1497975 2024-03-04T17:50:24.847 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.848+0000 7fe960f568c0 1 < completed write operation key=UEVMIPQ9FR81 tag=E6Y7ZLO44JZP type=write size=474276 2024-03-04T17:50:24.851 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.852+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=94KRYG1KO5KA type=del size=2624080 2024-03-04T17:50:24.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.856+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=B60TBFHXVP6B type=del 2024-03-04T17:50:24.859 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.860+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=RBTC390LXMO0 type=del size=1717633 2024-03-04T17:50:24.867 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.864+0000 7fe960f568c0 1 < suggested remove operation key=8PZD01OOX3AF tag=00XOFYNTTLZO type=write 2024-03-04T17:50:24.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.872+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~MSZ02195TECZ.0 size=1048576 2024-03-04T17:50:24.876 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.876+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~MSZ02195TECZ.1 size=1048576 2024-03-04T17:50:24.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.876+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~MSZ02195TECZ.2 size=290285 2024-03-04T17:50:24.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.876+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=MSZ02195TECZ tag=FO7KZ00974IK type=write size=3436013 2024-03-04T17:50:24.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.880+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~GGBTV3S02UY0.0 size=1048576 2024-03-04T17:50:24.886 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.884+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~GGBTV3S02UY0.1 size=1048576 2024-03-04T17:50:24.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.888+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~GGBTV3S02UY0.2 size=82542 2024-03-04T17:50:24.895 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.892+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=GGBTV3S02UY0 tag=F32KV646I9RS type=write size=3228270 2024-03-04T17:50:24.899 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.900+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=H9UML93V3CFW type=del size=2204564 2024-03-04T17:50:24.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.904+0000 7fe960f568c0 1 < completed write operation key=WNXVI6K5MU3R tag=KUNAEQ0R4S2U type=write size=159392 2024-03-04T17:50:24.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.908+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~BI4X1OY78U92.0 size=1048576 2024-03-04T17:50:24.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.912+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~BI4X1OY78U92.1 size=563905 2024-03-04T17:50:24.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.912+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=BI4X1OY78U92 tag=Q66U3EQDH9DS type=write size=2661057 2024-03-04T17:50:24.917 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.916+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~89END4VAJ5LI.0 size=1048576 2024-03-04T17:50:24.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.920+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~89END4VAJ5LI.1 size=1048576 2024-03-04T17:50:24.924 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.924+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~89END4VAJ5LI.2 size=523391 2024-03-04T17:50:24.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.928+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=89END4VAJ5LI tag=0COUJ5PV7VSY type=write size=3669119 2024-03-04T17:50:24.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.936+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~0NYLTZ1TB6DS.0 size=1048576 2024-03-04T17:50:24.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.940+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~0NYLTZ1TB6DS.1 size=1048576 2024-03-04T17:50:24.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.944+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~0NYLTZ1TB6DS.2 size=204439 2024-03-04T17:50:24.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.944+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=0NYLTZ1TB6DS tag=YBZIKR5LO7V7 type=write size=3350167 2024-03-04T17:50:24.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.952+0000 7fe960f568c0 1 < suggested remove operation key=NF4OIJWJ83DE tag=OM35EBN23BTT type=del 2024-03-04T17:50:24.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.956+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~VXH70Y0K4D75.0 size=1048576 2024-03-04T17:50:24.960 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.960+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~VXH70Y0K4D75.1 size=812148 2024-03-04T17:50:24.960 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.960+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=VXH70Y0K4D75 tag=G5N31Q2NG37V type=write size=2909300 2024-03-04T17:50:24.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.960+0000 7fe960f568c0 1 < canceled part key=_multipart_UEVMIPQ9FR81.2~HVDQP468R6HT.0 size=85461 2024-03-04T17:50:24.967 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.964+0000 7fe960f568c0 1 < canceled multipart upload key=UEVMIPQ9FR81 upload=HVDQP468R6HT tag=I86AKN3FZSGM type=write size=1134037 2024-03-04T17:50:24.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.972+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~B5AQGDH8QYGJ.0 size=211096 2024-03-04T17:50:24.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.972+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=B5AQGDH8QYGJ tag=YIO118W0BXQT type=write size=1259672 2024-03-04T17:50:24.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.976+0000 7fe960f568c0 -1 < failed to complete operation key=8PZD01OOX3AF tag=DDH3T4MBHXN7 type=del size=2224949: (22) Invalid argument 2024-03-04T17:50:24.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.980+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=ANFJRZ1TEDKU type=del size=3401123 2024-03-04T17:50:24.985 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.984+0000 7fe960f568c0 1 < canceled operation key=BFF9OD00TU54 tag=GMFB696KJJXY type=write size=211459 2024-03-04T17:50:24.989 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.988+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=19X4JBKN6T6A type=del size=3544569 2024-03-04T17:50:24.992 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.992+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~V1SNP6EW5273.0 size=807943 2024-03-04T17:50:24.998 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:24.996+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=V1SNP6EW5273 tag=F1N9Y2X0R3TO type=write size=1856519 2024-03-04T17:50:25.002 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.000+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=G5FIWGCAYTAK type=del size=4038955 2024-03-04T17:50:25.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.004+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~C62QCLN5QEWP.0 size=463598 2024-03-04T17:50:25.013 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.012+0000 7fe960f568c0 1 < completed multipart upload key=2MKQA5JYWU8L upload=C62QCLN5QEWP tag=6Z9VLUNSQXJY type=write size=1512174 2024-03-04T17:50:25.019 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.020+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~JU6V121R9B82.0 size=640637 2024-03-04T17:50:25.019 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.020+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=JU6V121R9B82 tag=IJPX1VAJRMME type=write size=1689213 2024-03-04T17:50:25.023 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.020+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~RUKTFXAJLX18.0 size=1048576 2024-03-04T17:50:25.023 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.024+0000 7fe960f568c0 1 < canceled part key=_multipart_MIZRUP4MEA0J.2~RUKTFXAJLX18.1 size=56275 2024-03-04T17:50:25.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.032+0000 7fe960f568c0 1 < canceled multipart upload key=MIZRUP4MEA0J upload=RUKTFXAJLX18 tag=1V1BYMV0UYEP type=write size=2153427 2024-03-04T17:50:25.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.036+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=405JNZSFWWR5 type=del size=448221 2024-03-04T17:50:25.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.040+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=GCSUPTP2I5B2 type=del 2024-03-04T17:50:25.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.044+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~LDRXH5BZ894C.0 size=1048576 2024-03-04T17:50:25.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.048+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~LDRXH5BZ894C.1 size=1048576 2024-03-04T17:50:25.052 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.052+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~LDRXH5BZ894C.2 size=137732 2024-03-04T17:50:25.052 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.052+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=LDRXH5BZ894C tag=EH99UDBPFTAA type=write size=3283460 2024-03-04T17:50:25.056 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.056+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~N34RW0DWURTR.0 size=547553 2024-03-04T17:50:25.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.060+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=N34RW0DWURTR tag=WLFLKAGJMDAW type=write size=1596129 2024-03-04T17:50:25.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.068+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~H908U477RPNM.0 size=1048576 2024-03-04T17:50:25.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.072+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~H908U477RPNM.1 size=1048576 2024-03-04T17:50:25.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.076+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~H908U477RPNM.2 size=291661 2024-03-04T17:50:25.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.076+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=H908U477RPNM tag=VZEJOPGNFWXU type=write size=3437389 2024-03-04T17:50:25.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.080+0000 7fe960f568c0 1 < canceled operation key=78N8HHC4O4CG tag=E3YPK6LLILRI type=del size=1497975 2024-03-04T17:50:25.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.084+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~29QRPOYWYNZ2.0 size=1048576 2024-03-04T17:50:25.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.088+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~29QRPOYWYNZ2.1 size=1048576 2024-03-04T17:50:25.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.092+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~29QRPOYWYNZ2.2 size=990129 2024-03-04T17:50:25.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.092+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=29QRPOYWYNZ2 tag=P91JKNSGH6L5 type=write size=4135857 2024-03-04T17:50:25.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.096+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=94KRYG1KO5KA type=del 2024-03-04T17:50:25.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.100+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=YXPIX022SEV8 type=write size=1011026 2024-03-04T17:50:25.104 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.104+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=RBTC390LXMO0 type=del 2024-03-04T17:50:25.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.108+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=SBAA2SYKDMKG type=del size=2089081 2024-03-04T17:50:25.112 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.112+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~MSZ02195TECZ.0 size=1048576 2024-03-04T17:50:25.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.116+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~MSZ02195TECZ.1 size=1048576 2024-03-04T17:50:25.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.116+0000 7fe960f568c0 1 < canceled part key=_multipart_2MKQA5JYWU8L.2~MSZ02195TECZ.2 size=290285 2024-03-04T17:50:25.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.120+0000 7fe960f568c0 1 < canceled multipart upload key=2MKQA5JYWU8L upload=MSZ02195TECZ tag=FO7KZ00974IK type=write size=3436013 2024-03-04T17:50:25.128 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.128+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~QZL6AREZ3LQ2.0 size=1048576 2024-03-04T17:50:25.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.132+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~QZL6AREZ3LQ2.1 size=1048576 2024-03-04T17:50:25.134 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.132+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~QZL6AREZ3LQ2.2 size=530890 2024-03-04T17:50:25.134 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.132+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=QZL6AREZ3LQ2 tag=6YS71A970RPZ type=write size=3676618 2024-03-04T17:50:25.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.136+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=H9UML93V3CFW type=del 2024-03-04T17:50:25.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.144+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~CWWLD2BX4MFL.0 size=259128 2024-03-04T17:50:25.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.144+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=CWWLD2BX4MFL tag=VA1ZMQGD4K51 type=write size=1307704 2024-03-04T17:50:25.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.148+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~BI4X1OY78U92.0 size=1048576 2024-03-04T17:50:25.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.152+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~BI4X1OY78U92.1 size=563905 2024-03-04T17:50:25.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.156+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=BI4X1OY78U92 tag=Q66U3EQDH9DS type=write size=2661057 2024-03-04T17:50:25.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.160+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=F91CEXJREW55 type=del size=3378469 2024-03-04T17:50:25.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.164+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~0NYLTZ1TB6DS.0 size=1048576 2024-03-04T17:50:25.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.168+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~0NYLTZ1TB6DS.1 size=1048576 2024-03-04T17:50:25.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.172+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~0NYLTZ1TB6DS.2 size=204439 2024-03-04T17:50:25.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.176+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=0NYLTZ1TB6DS tag=YBZIKR5LO7V7 type=write size=3350167 2024-03-04T17:50:25.181 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.180+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=L13SAGC208FJ type=del size=1959194 2024-03-04T17:50:25.188 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.188+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~VXH70Y0K4D75.0 size=1048576 2024-03-04T17:50:25.191 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.192+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~VXH70Y0K4D75.1 size=812148 2024-03-04T17:50:25.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.196+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=VXH70Y0K4D75 tag=G5N31Q2NG37V type=write size=2909300 2024-03-04T17:50:25.203 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.200+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=C1J12S8W3GQN type=del size=3332280 2024-03-04T17:50:25.208 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.208+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~B5AQGDH8QYGJ.0 size=211096 2024-03-04T17:50:25.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.212+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=B5AQGDH8QYGJ tag=YIO118W0BXQT type=write size=1259672 2024-03-04T17:50:25.218 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.216+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=IGWASEDNXENQ type=del size=1948648 2024-03-04T17:50:25.226 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.224+0000 7fe960f568c0 1 < suggested update operation key=3X4719GHK3BI tag=ANFJRZ1TEDKU type=del size=3401123 2024-03-04T17:50:25.233 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.232+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~R81TEMTWTLMQ.0 size=1048576 2024-03-04T17:50:25.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.236+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~R81TEMTWTLMQ.1 size=1048576 2024-03-04T17:50:25.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.236+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~R81TEMTWTLMQ.2 size=675644 2024-03-04T17:50:25.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.236+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=R81TEMTWTLMQ tag=23PN8DU8PBFB type=write size=3821372 2024-03-04T17:50:25.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.240+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=19X4JBKN6T6A type=del 2024-03-04T17:50:25.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.248+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~CVXAA3LPQF45.0 size=1048576 2024-03-04T17:50:25.251 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.252+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~CVXAA3LPQF45.1 size=1048576 2024-03-04T17:50:25.254 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.252+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~CVXAA3LPQF45.2 size=92842 2024-03-04T17:50:25.254 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.252+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=CVXAA3LPQF45 tag=J1MB4DQC0FVU type=write size=3238570 2024-03-04T17:50:25.258 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.256+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=G5FIWGCAYTAK type=del 2024-03-04T17:50:25.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.264+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~PUZSN3WE23JB.0 size=1048576 2024-03-04T17:50:25.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.268+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~PUZSN3WE23JB.1 size=1048576 2024-03-04T17:50:25.270 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.268+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~PUZSN3WE23JB.2 size=795990 2024-03-04T17:50:25.270 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.268+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=PUZSN3WE23JB tag=ZCS36R6YND53 type=write size=3941718 2024-03-04T17:50:25.271 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.268+0000 7fe960f568c0 1 < canceled part key=_multipart_QFF67JIW8Y8E.2~JU6V121R9B82.0 size=640637 2024-03-04T17:50:25.277 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.276+0000 7fe960f568c0 1 < canceled multipart upload key=QFF67JIW8Y8E upload=JU6V121R9B82 tag=IJPX1VAJRMME type=write size=1689213 2024-03-04T17:50:25.281 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.280+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=1FB9UV6QQLE4 type=write size=409830 2024-03-04T17:50:25.285 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.284+0000 7fe960f568c0 1 < canceled operation key=3YTAYUQLLUKY tag=405JNZSFWWR5 type=del size=448221 2024-03-04T17:50:25.292 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.292+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~O6MW2NQLIKP6.0 size=1048576 2024-03-04T17:50:25.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.292+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~O6MW2NQLIKP6.1 size=1048576 2024-03-04T17:50:25.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.296+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~O6MW2NQLIKP6.2 size=421000 2024-03-04T17:50:25.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.296+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=O6MW2NQLIKP6 tag=4IPUST8KXCZI type=write size=3566728 2024-03-04T17:50:25.301 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.300+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~LDRXH5BZ894C.0 size=1048576 2024-03-04T17:50:25.304 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.304+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~LDRXH5BZ894C.1 size=1048576 2024-03-04T17:50:25.307 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.308+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~LDRXH5BZ894C.2 size=137732 2024-03-04T17:50:25.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.312+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=LDRXH5BZ894C tag=EH99UDBPFTAA type=write size=3283460 2024-03-04T17:50:25.317 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.316+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=ZHJSAN9F9YE7 type=write size=152750 2024-03-04T17:50:25.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.320+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~H908U477RPNM.0 size=1048576 2024-03-04T17:50:25.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.324+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~H908U477RPNM.1 size=1048576 2024-03-04T17:50:25.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.328+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~H908U477RPNM.2 size=291661 2024-03-04T17:50:25.335 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.332+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=H908U477RPNM tag=VZEJOPGNFWXU type=write size=3437389 2024-03-04T17:50:25.339 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.336+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=89PX9TJMDMNA type=del size=2307358 2024-03-04T17:50:25.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.340+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~29QRPOYWYNZ2.0 size=1048576 2024-03-04T17:50:25.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.344+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~29QRPOYWYNZ2.1 size=1048576 2024-03-04T17:50:25.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.348+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~29QRPOYWYNZ2.2 size=990129 2024-03-04T17:50:25.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.356+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=29QRPOYWYNZ2 tag=P91JKNSGH6L5 type=write size=4135857 2024-03-04T17:50:25.359 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.360+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=69JYZ8G4G73Q type=del size=284389 2024-03-04T17:50:25.363 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.364+0000 7fe960f568c0 1 < completed write operation key=GZ4RRIR5CC26 tag=YXPIX022SEV8 type=write size=1011026 2024-03-04T17:50:25.367 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.364+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=UCY8BGXP6AM3 type=del size=696473 2024-03-04T17:50:25.371 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.372+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=SBAA2SYKDMKG type=del 2024-03-04T17:50:25.375 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.372+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=KW75OC5AVIME type=del size=583948 2024-03-04T17:50:25.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.376+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~QZL6AREZ3LQ2.0 size=1048576 2024-03-04T17:50:25.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.380+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~QZL6AREZ3LQ2.1 size=1048576 2024-03-04T17:50:25.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.384+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~QZL6AREZ3LQ2.2 size=530890 2024-03-04T17:50:25.391 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.388+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=QZL6AREZ3LQ2 tag=6YS71A970RPZ type=write size=3676618 2024-03-04T17:50:25.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.392+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=3F767S7S1UB5 type=write size=9641 2024-03-04T17:50:25.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.396+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~CWWLD2BX4MFL.0 size=259128 2024-03-04T17:50:25.405 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.404+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=CWWLD2BX4MFL tag=VA1ZMQGD4K51 type=write size=1307704 2024-03-04T17:50:25.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.408+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=5RERBZTMUZ6X type=del size=2318632 2024-03-04T17:50:25.412 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.412+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=F91CEXJREW55 type=del 2024-03-04T17:50:25.415 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.416+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=P1WDPGELXB60 type=del size=3157443 2024-03-04T17:50:25.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.420+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=L13SAGC208FJ type=del 2024-03-04T17:50:25.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.424+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~JSGJKOFXI1PE.0 size=541305 2024-03-04T17:50:25.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.424+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=JSGJKOFXI1PE tag=YKA2FDRR5VUJ type=write size=1589881 2024-03-04T17:50:25.430 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.428+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=C1J12S8W3GQN type=del 2024-03-04T17:50:25.437 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.436+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~DQY3HZUMBI2M.0 size=1048576 2024-03-04T17:50:25.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.440+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~DQY3HZUMBI2M.1 size=1048576 2024-03-04T17:50:25.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.440+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~DQY3HZUMBI2M.2 size=858728 2024-03-04T17:50:25.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.440+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=DQY3HZUMBI2M tag=GKODLVTPC771 type=write size=4004456 2024-03-04T17:50:25.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.444+0000 7fe960f568c0 1 < completed delete operation key=U2G6YEVD5506 tag=IGWASEDNXENQ type=del 2024-03-04T17:50:25.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.452+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~PCA321GEF75M.0 size=725249 2024-03-04T17:50:25.454 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.452+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=PCA321GEF75M tag=4OQO1VPI9WYM type=write size=1773825 2024-03-04T17:50:25.454 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.452+0000 7fe960f568c0 1 < canceled part key=_multipart_78N8HHC4O4CG.2~R81TEMTWTLMQ.0 size=1048576 2024-03-04T17:50:25.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.456+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~R81TEMTWTLMQ.1 size=1048576 2024-03-04T17:50:25.460 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.460+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~R81TEMTWTLMQ.2 size=675644 2024-03-04T17:50:25.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.464+0000 7fe960f568c0 1 < canceled multipart upload key=78N8HHC4O4CG upload=R81TEMTWTLMQ tag=23PN8DU8PBFB type=write size=3821372 2024-03-04T17:50:25.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.472+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~GRTL6SZ6J7SV.0 size=987538 2024-03-04T17:50:25.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.472+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=GRTL6SZ6J7SV tag=70705XLSPF5Z type=write size=2036114 2024-03-04T17:50:25.477 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.476+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~CVXAA3LPQF45.0 size=1048576 2024-03-04T17:50:25.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.480+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~CVXAA3LPQF45.1 size=1048576 2024-03-04T17:50:25.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.480+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~CVXAA3LPQF45.2 size=92842 2024-03-04T17:50:25.488 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.488+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=CVXAA3LPQF45 tag=J1MB4DQC0FVU type=write size=3238570 2024-03-04T17:50:25.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.492+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=G2GTMQ0AVG22 type=del size=138043 2024-03-04T17:50:25.493 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.492+0000 7fe960f568c0 1 < canceled part key=_multipart_UKZ079SVZZKG.2~PUZSN3WE23JB.0 size=1048576 2024-03-04T17:50:25.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.496+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~PUZSN3WE23JB.1 size=1048576 2024-03-04T17:50:25.499 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.496+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~PUZSN3WE23JB.2 size=795990 2024-03-04T17:50:25.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.504+0000 7fe960f568c0 1 < canceled multipart upload key=UKZ079SVZZKG upload=PUZSN3WE23JB tag=ZCS36R6YND53 type=write size=3941718 2024-03-04T17:50:25.512 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.512+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~R8ZPM5R3IAM4.0 size=902126 2024-03-04T17:50:25.512 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.512+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=R8ZPM5R3IAM4 tag=MWUUSZT2VAP2 type=write size=1950702 2024-03-04T17:50:25.516 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.516+0000 7fe960f568c0 1 < completed write operation key=3X4719GHK3BI tag=1FB9UV6QQLE4 type=write size=409830 2024-03-04T17:50:25.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.520+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=H75R51DVEZB8 type=del size=1792973 2024-03-04T17:50:25.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.524+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~O6MW2NQLIKP6.0 size=1048576 2024-03-04T17:50:25.528 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.528+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~O6MW2NQLIKP6.1 size=1048576 2024-03-04T17:50:25.535 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.536+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~O6MW2NQLIKP6.2 size=421000 2024-03-04T17:50:25.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.540+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=O6MW2NQLIKP6 tag=4IPUST8KXCZI type=write size=3566728 2024-03-04T17:50:25.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.544+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=VR9TI1WQUTY1 type=del size=4116591 2024-03-04T17:50:25.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.548+0000 7fe960f568c0 1 < completed write operation key=768X2XKWJW1S tag=ZHJSAN9F9YE7 type=write size=152750 2024-03-04T17:50:25.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.556+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~FR2L8FQPOZSE.0 size=1048576 2024-03-04T17:50:25.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.556+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~FR2L8FQPOZSE.1 size=1048576 2024-03-04T17:50:25.561 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.560+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~FR2L8FQPOZSE.2 size=332742 2024-03-04T17:50:25.561 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.560+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=FR2L8FQPOZSE tag=V425OUYL8J7G type=write size=3478470 2024-03-04T17:50:25.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.564+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=89PX9TJMDMNA type=del 2024-03-04T17:50:25.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.568+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=KDUFG3Q5EVTO type=write size=214653 2024-03-04T17:50:25.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.572+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=69JYZ8G4G73Q type=del 2024-03-04T17:50:25.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.576+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=F4CJP0BACPJW type=del size=410677 2024-03-04T17:50:25.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.580+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=UCY8BGXP6AM3 type=del 2024-03-04T17:50:25.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.584+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=YJWMRCF3JQDJ type=del size=4175992 2024-03-04T17:50:25.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.588+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=KW75OC5AVIME type=del 2024-03-04T17:50:25.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.592+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~EYI2NOYBQ1C2.0 size=1048576 2024-03-04T17:50:25.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.596+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~EYI2NOYBQ1C2.1 size=46464 2024-03-04T17:50:25.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.596+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=EYI2NOYBQ1C2 tag=AA3TGJECT3UU type=write size=2143616 2024-03-04T17:50:25.601 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.600+0000 7fe960f568c0 1 < completed write operation key=78N8HHC4O4CG tag=3F767S7S1UB5 type=write size=9641 2024-03-04T17:50:25.605 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.604+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=LPTXJ857PO49 type=del size=2457535 2024-03-04T17:50:25.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.608+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=5RERBZTMUZ6X type=del 2024-03-04T17:50:25.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.612+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=H01W2FDERY0L type=write size=919881 2024-03-04T17:50:25.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.616+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=P1WDPGELXB60 type=del 2024-03-04T17:50:25.623 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.620+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~RXOWHF29XV4C.0 size=406043 2024-03-04T17:50:25.623 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.620+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=RXOWHF29XV4C tag=9FF49CIW3JDW type=write size=1454619 2024-03-04T17:50:25.626 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.624+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~JSGJKOFXI1PE.0 size=541305 2024-03-04T17:50:25.632 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.632+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=JSGJKOFXI1PE tag=YKA2FDRR5VUJ type=write size=1589881 2024-03-04T17:50:25.636 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.636+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=IYXH2QNQUV6C type=del size=302125 2024-03-04T17:50:25.640 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.640+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~DQY3HZUMBI2M.0 size=1048576 2024-03-04T17:50:25.643 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.644+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~DQY3HZUMBI2M.1 size=1048576 2024-03-04T17:50:25.646 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.644+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~DQY3HZUMBI2M.2 size=858728 2024-03-04T17:50:25.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.652+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=DQY3HZUMBI2M tag=GKODLVTPC771 type=write size=4004456 2024-03-04T17:50:25.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.656+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~8C1TSM9FE9CW.0 size=821021 2024-03-04T17:50:25.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.656+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=8C1TSM9FE9CW tag=CC8QVIALKQOW type=write size=1869597 2024-03-04T17:50:25.660 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.660+0000 7fe960f568c0 1 < canceled part key=_multipart_UEVMIPQ9FR81.2~PCA321GEF75M.0 size=725249 2024-03-04T17:50:25.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.664+0000 7fe960f568c0 1 < canceled multipart upload key=UEVMIPQ9FR81 upload=PCA321GEF75M tag=4OQO1VPI9WYM type=write size=1773825 2024-03-04T17:50:25.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.668+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=JL3UN7SGTB9V type=del size=1339988 2024-03-04T17:50:25.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.672+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~GRTL6SZ6J7SV.0 size=987538 2024-03-04T17:50:25.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.680+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=GRTL6SZ6J7SV tag=70705XLSPF5Z type=write size=2036114 2024-03-04T17:50:25.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.684+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=R8KCOYU1HV1C type=write size=393362 2024-03-04T17:50:25.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.688+0000 7fe960f568c0 1 < suggested update operation key=FLYDZHOIWO6L tag=G2GTMQ0AVG22 type=del size=138043 2024-03-04T17:50:25.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.696+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~YMTS51IWKFZB.0 size=1048576 2024-03-04T17:50:25.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.700+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~YMTS51IWKFZB.1 size=1048576 2024-03-04T17:50:25.702 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.700+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~YMTS51IWKFZB.2 size=591435 2024-03-04T17:50:25.702 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.700+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=YMTS51IWKFZB tag=ZHP9YKE31I4R type=write size=3737163 2024-03-04T17:50:25.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.704+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~R8ZPM5R3IAM4.0 size=902126 2024-03-04T17:50:25.712 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.712+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=R8ZPM5R3IAM4 tag=MWUUSZT2VAP2 type=write size=1950702 2024-03-04T17:50:25.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.716+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~AHI2EHT4ABSG.0 size=1048576 2024-03-04T17:50:25.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.720+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~AHI2EHT4ABSG.1 size=922519 2024-03-04T17:50:25.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.720+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=AHI2EHT4ABSG tag=B83S97MS63L1 type=write size=3019671 2024-03-04T17:50:25.726 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.724+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=H75R51DVEZB8 type=del 2024-03-04T17:50:25.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.732+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~3Q04DZK9ZUJK.0 size=34960 2024-03-04T17:50:25.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.732+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=3Q04DZK9ZUJK tag=4T1PQ23HC0W3 type=write size=1083536 2024-03-04T17:50:25.736 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.736+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=VR9TI1WQUTY1 type=del 2024-03-04T17:50:25.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.744+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~GYR63878JW9N.0 size=1048576 2024-03-04T17:50:25.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.744+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~GYR63878JW9N.1 size=867893 2024-03-04T17:50:25.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.744+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=GYR63878JW9N tag=63MGCER2L5KO type=write size=2965045 2024-03-04T17:50:25.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.748+0000 7fe960f568c0 1 < canceled part key=_multipart_MIZRUP4MEA0J.2~FR2L8FQPOZSE.0 size=1048576 2024-03-04T17:50:25.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.748+0000 7fe960f568c0 1 < canceled part key=_multipart_MIZRUP4MEA0J.2~FR2L8FQPOZSE.1 size=1048576 2024-03-04T17:50:25.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.748+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~FR2L8FQPOZSE.2 size=332742 2024-03-04T17:50:25.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.756+0000 7fe960f568c0 1 < canceled multipart upload key=MIZRUP4MEA0J upload=FR2L8FQPOZSE tag=V425OUYL8J7G type=write size=3478470 2024-03-04T17:50:25.760 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.760+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=OABQU1SMI3DW type=del size=333262 2024-03-04T17:50:25.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.764+0000 7fe960f568c0 1 < completed write operation key=UEVMIPQ9FR81 tag=KDUFG3Q5EVTO type=write size=214653 2024-03-04T17:50:25.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.768+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=KBRYLB1TJOQS type=del size=79992 2024-03-04T17:50:25.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.772+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=F4CJP0BACPJW type=del 2024-03-04T17:50:25.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.776+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~U71FD7CN9VTG.0 size=30991 2024-03-04T17:50:25.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.776+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=U71FD7CN9VTG tag=05JO2IETDB74 type=write size=1079567 2024-03-04T17:50:25.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.780+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=YJWMRCF3JQDJ type=del 2024-03-04T17:50:25.787 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.784+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=GWO024HE0T52 type=del size=4128119 2024-03-04T17:50:25.791 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.788+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~EYI2NOYBQ1C2.0 size=1048576 2024-03-04T17:50:25.793 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.792+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~EYI2NOYBQ1C2.1 size=46464 2024-03-04T17:50:25.799 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.800+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=EYI2NOYBQ1C2 tag=AA3TGJECT3UU type=write size=2143616 2024-03-04T17:50:25.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.804+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=LVBWFYW8BXO3 type=write size=791159 2024-03-04T17:50:25.808 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.808+0000 7fe960f568c0 1 < canceled operation key=7Z7BPZ8XGBHZ tag=LPTXJ857PO49 type=del size=2457535 2024-03-04T17:50:25.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.812+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=MOG6YHD6U54A type=del size=3995764 2024-03-04T17:50:25.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.816+0000 7fe960f568c0 1 < suggested update operation key=JJ7SM1Y73A4C tag=H01W2FDERY0L type=write size=919881 2024-03-04T17:50:25.822 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.820+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=0I0IGQ7JIZBF type=write size=111570 2024-03-04T17:50:25.826 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.824+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~RXOWHF29XV4C.0 size=406043 2024-03-04T17:50:25.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.832+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=RXOWHF29XV4C tag=9FF49CIW3JDW type=write size=1454619 2024-03-04T17:50:25.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.836+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~PNN0417YJVWE.0 size=1048576 2024-03-04T17:50:25.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.840+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~PNN0417YJVWE.1 size=1048576 2024-03-04T17:50:25.844 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.844+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~PNN0417YJVWE.2 size=257530 2024-03-04T17:50:25.844 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.844+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=PNN0417YJVWE tag=Z3LW3ZLZC1PA type=write size=3403258 2024-03-04T17:50:25.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.848+0000 7fe960f568c0 1 < completed delete operation key=U2G6YEVD5506 tag=IYXH2QNQUV6C type=del 2024-03-04T17:50:25.853 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.852+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=SNE16LZHOOIQ type=del size=3357668 2024-03-04T17:50:25.857 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.856+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~8C1TSM9FE9CW.0 size=821021 2024-03-04T17:50:25.863 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.860+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=8C1TSM9FE9CW tag=CC8QVIALKQOW type=write size=1869597 2024-03-04T17:50:25.867 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.864+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=F0HPLKC7012Y type=del size=2123735 2024-03-04T17:50:25.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.868+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=JL3UN7SGTB9V type=del 2024-03-04T17:50:25.874 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.872+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=YVKLK7M63UXB type=del size=1484401 2024-03-04T17:50:25.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.876+0000 7fe960f568c0 1 < completed write operation key=8PZD01OOX3AF tag=R8KCOYU1HV1C type=write size=393362 2024-03-04T17:50:25.886 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.884+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=1PLG6IZUID1U type=write size=1004769 2024-03-04T17:50:25.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.888+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~YMTS51IWKFZB.0 size=1048576 2024-03-04T17:50:25.893 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.892+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~YMTS51IWKFZB.1 size=1048576 2024-03-04T17:50:25.896 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.896+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~YMTS51IWKFZB.2 size=591435 2024-03-04T17:50:25.902 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.900+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=YMTS51IWKFZB tag=ZHP9YKE31I4R type=write size=3737163 2024-03-04T17:50:25.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.908+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~IM9M19OBTLZ2.0 size=1045565 2024-03-04T17:50:25.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.908+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=IM9M19OBTLZ2 tag=OBA38HJ0NEVF type=write size=2094141 2024-03-04T17:50:25.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.908+0000 7fe960f568c0 1 < canceled part key=_multipart_NF4OIJWJ83DE.2~AHI2EHT4ABSG.0 size=1048576 2024-03-04T17:50:25.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.912+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~AHI2EHT4ABSG.1 size=922519 2024-03-04T17:50:25.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.920+0000 7fe960f568c0 1 < canceled multipart upload key=NF4OIJWJ83DE upload=AHI2EHT4ABSG tag=B83S97MS63L1 type=write size=3019671 2024-03-04T17:50:25.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.920+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=1JLGWXBFVTV9 type=del size=1749618 2024-03-04T17:50:25.926 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.924+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~3Q04DZK9ZUJK.0 size=34960 2024-03-04T17:50:25.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.932+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=3Q04DZK9ZUJK tag=4T1PQ23HC0W3 type=write size=1083536 2024-03-04T17:50:25.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.936+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~54TA09RRNA2L.0 size=1048576 2024-03-04T17:50:25.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.940+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~54TA09RRNA2L.1 size=1048576 2024-03-04T17:50:25.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.944+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~54TA09RRNA2L.2 size=657683 2024-03-04T17:50:25.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.944+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=54TA09RRNA2L tag=ZTFCT9IYJLG1 type=write size=3803411 2024-03-04T17:50:25.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.948+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~GYR63878JW9N.0 size=1048576 2024-03-04T17:50:25.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.952+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~GYR63878JW9N.1 size=867893 2024-03-04T17:50:25.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.956+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=GYR63878JW9N tag=63MGCER2L5KO type=write size=2965045 2024-03-04T17:50:25.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.960+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=WZIAX00Z179Q type=del size=601873 2024-03-04T17:50:25.965 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.964+0000 7fe960f568c0 1 < canceled operation key=NF4OIJWJ83DE tag=OABQU1SMI3DW type=del size=333262 2024-03-04T17:50:25.969 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.968+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=HUJXZH7URPMD type=del size=3123699 2024-03-04T17:50:25.973 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.972+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=KBRYLB1TJOQS type=del 2024-03-04T17:50:25.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.976+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=9RE2NDP9L1V8 type=del size=4054938 2024-03-04T17:50:25.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.980+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~U71FD7CN9VTG.0 size=30991 2024-03-04T17:50:25.987 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.988+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=U71FD7CN9VTG tag=05JO2IETDB74 type=write size=1079567 2024-03-04T17:50:25.992 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.992+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=JIS2ZIAG1KFG type=del size=790507 2024-03-04T17:50:25.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:25.996+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=GWO024HE0T52 type=del 2024-03-04T17:50:26.003 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.004+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~PAKUCZTDAHRN.0 size=1048576 2024-03-04T17:50:26.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.004+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~PAKUCZTDAHRN.1 size=70623 2024-03-04T17:50:26.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.004+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=PAKUCZTDAHRN tag=SWQTHUQXKRSV type=write size=2167775 2024-03-04T17:50:26.010 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.008+0000 7fe960f568c0 1 < completed write operation key=3YTAYUQLLUKY tag=LVBWFYW8BXO3 type=write size=791159 2024-03-04T17:50:26.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.012+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=JCPWTBLCTTWX type=del size=1100333 2024-03-04T17:50:26.018 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.016+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=MOG6YHD6U54A type=del 2024-03-04T17:50:26.022 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.020+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=JEEFTU1XLYOQ type=del size=2338067 2024-03-04T17:50:26.026 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.024+0000 7fe960f568c0 1 < completed write operation key=JT02WL9JYTNM tag=0I0IGQ7JIZBF type=write size=111570 2024-03-04T17:50:26.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.032+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~EQ0ZSXMVM0NJ.0 size=1048576 2024-03-04T17:50:26.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.036+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~EQ0ZSXMVM0NJ.1 size=157582 2024-03-04T17:50:26.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.036+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=EQ0ZSXMVM0NJ tag=F6577F2PSU10 type=write size=2254734 2024-03-04T17:50:26.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.040+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~PNN0417YJVWE.0 size=1048576 2024-03-04T17:50:26.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.040+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~PNN0417YJVWE.1 size=1048576 2024-03-04T17:50:26.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.044+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~PNN0417YJVWE.2 size=257530 2024-03-04T17:50:26.051 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.052+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=PNN0417YJVWE tag=Z3LW3ZLZC1PA type=write size=3403258 2024-03-04T17:50:26.055 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.052+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=9UHZZYJJJEM2 type=del size=440665 2024-03-04T17:50:26.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.056+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=SNE16LZHOOIQ type=del 2024-03-04T17:50:26.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.064+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~YFX41U1ROQ0E.0 size=113060 2024-03-04T17:50:26.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.064+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=YFX41U1ROQ0E tag=LNQZLJKVWGR3 type=write size=1161636 2024-03-04T17:50:26.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.068+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=F0HPLKC7012Y type=del 2024-03-04T17:50:26.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.072+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~WOQLPE3ZPOO7.0 size=130342 2024-03-04T17:50:26.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.072+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=WOQLPE3ZPOO7 tag=L3CEU917GAG6 type=write size=1178918 2024-03-04T17:50:26.078 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.076+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=YVKLK7M63UXB type=del 2024-03-04T17:50:26.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.084+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~67YFVPN7O69V.0 size=1048576 2024-03-04T17:50:26.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.088+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~67YFVPN7O69V.1 size=749241 2024-03-04T17:50:26.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.088+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=67YFVPN7O69V tag=KKALFTWM0OOR type=write size=2846393 2024-03-04T17:50:26.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.092+0000 7fe960f568c0 1 < completed write operation key=QGLN27WH1N2O tag=1PLG6IZUID1U type=write size=1004769 2024-03-04T17:50:26.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.096+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=HJUA74JW8JUN type=del size=236297 2024-03-04T17:50:26.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.096+0000 7fe960f568c0 1 < canceled part key=_multipart_JP3FRB6MD7B4.2~IM9M19OBTLZ2.0 size=1045565 2024-03-04T17:50:26.103 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.100+0000 7fe960f568c0 1 < canceled multipart upload key=JP3FRB6MD7B4 upload=IM9M19OBTLZ2 tag=OBA38HJ0NEVF type=write size=2094141 2024-03-04T17:50:26.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.104+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=YK2T3R1U3DBB type=del size=3506458 2024-03-04T17:50:26.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.112+0000 7fe960f568c0 1 < suggested remove operation key=UEVMIPQ9FR81 tag=1JLGWXBFVTV9 type=del 2024-03-04T17:50:26.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.120+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~F3G9PPHKCLVW.0 size=136797 2024-03-04T17:50:26.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.120+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=F3G9PPHKCLVW tag=1E1DX3MSH8JE type=write size=1185373 2024-03-04T17:50:26.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.124+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~54TA09RRNA2L.0 size=1048576 2024-03-04T17:50:26.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.128+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~54TA09RRNA2L.1 size=1048576 2024-03-04T17:50:26.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.132+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~54TA09RRNA2L.2 size=657683 2024-03-04T17:50:26.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.136+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=54TA09RRNA2L tag=ZTFCT9IYJLG1 type=write size=3803411 2024-03-04T17:50:26.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.144+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~VRXT1GL3RZBP.0 size=1048576 2024-03-04T17:50:26.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.148+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~VRXT1GL3RZBP.1 size=578784 2024-03-04T17:50:26.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.148+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=VRXT1GL3RZBP tag=LAMVDX68L8H6 type=write size=2675936 2024-03-04T17:50:26.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.152+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=WZIAX00Z179Q type=del 2024-03-04T17:50:26.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.156+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~M3LAIL1OHATM.0 size=862297 2024-03-04T17:50:26.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.156+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=M3LAIL1OHATM tag=E4BVJ2Q8FC26 type=write size=1910873 2024-03-04T17:50:26.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.164+0000 7fe960f568c0 1 < suggested update operation key=U2G6YEVD5506 tag=HUJXZH7URPMD type=del size=3123699 2024-03-04T17:50:26.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.172+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~VG26DBHNHKFP.0 size=1048576 2024-03-04T17:50:26.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.176+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~VG26DBHNHKFP.1 size=740211 2024-03-04T17:50:26.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.176+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=VG26DBHNHKFP tag=46EXVDQIGGBL type=write size=2837363 2024-03-04T17:50:26.180 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.180+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=9RE2NDP9L1V8 type=del 2024-03-04T17:50:26.184 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.184+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=0RY8JAJTAIQA type=write size=316119 2024-03-04T17:50:26.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.184+0000 7fe960f568c0 -1 < no bi entry to suggest for operation key=UEVMIPQ9FR81 tag=JIS2ZIAG1KFG type=del size=790507: (2) No such file or directory 2024-03-04T17:50:26.190 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.188+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=KAP3X3RRC1ED type=del size=1507223 2024-03-04T17:50:26.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.192+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~PAKUCZTDAHRN.0 size=1048576 2024-03-04T17:50:26.197 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.196+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~PAKUCZTDAHRN.1 size=70623 2024-03-04T17:50:26.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.200+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=PAKUCZTDAHRN tag=SWQTHUQXKRSV type=write size=2167775 2024-03-04T17:50:26.206 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.204+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=K2QP473QLJGN type=del size=520864 2024-03-04T17:50:26.210 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.208+0000 7fe960f568c0 1 < canceled operation key=FLYDZHOIWO6L tag=JCPWTBLCTTWX type=del size=1100333 2024-03-04T17:50:26.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.212+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=YSQSNR6E77IP type=write size=173939 2024-03-04T17:50:26.217 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.216+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=JEEFTU1XLYOQ type=del 2024-03-04T17:50:26.221 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.220+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=5OOZKJ6FZZU0 type=write size=17706 2024-03-04T17:50:26.225 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.224+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~EQ0ZSXMVM0NJ.0 size=1048576 2024-03-04T17:50:26.228 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.228+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~EQ0ZSXMVM0NJ.1 size=157582 2024-03-04T17:50:26.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.236+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=EQ0ZSXMVM0NJ tag=F6577F2PSU10 type=write size=2254734 2024-03-04T17:50:26.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.240+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=XH0K3K3LYCLA type=del size=1149364 2024-03-04T17:50:26.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.244+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=9UHZZYJJJEM2 type=del 2024-03-04T17:50:26.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.248+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=7FJN0EUUPH3T type=del size=2600630 2024-03-04T17:50:26.253 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.252+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~YFX41U1ROQ0E.0 size=113060 2024-03-04T17:50:26.259 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.256+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=YFX41U1ROQ0E tag=LNQZLJKVWGR3 type=write size=1161636 2024-03-04T17:50:26.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.260+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=AZJZERIRKW4H type=del size=89943 2024-03-04T17:50:26.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.264+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~WOQLPE3ZPOO7.0 size=130342 2024-03-04T17:50:26.271 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.272+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=WOQLPE3ZPOO7 tag=L3CEU917GAG6 type=write size=1178918 2024-03-04T17:50:26.275 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.272+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=28HO8O4UIEOH type=del size=3780885 2024-03-04T17:50:26.278 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.276+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~67YFVPN7O69V.0 size=1048576 2024-03-04T17:50:26.281 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.280+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~67YFVPN7O69V.1 size=749241 2024-03-04T17:50:26.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.284+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=67YFVPN7O69V tag=KKALFTWM0OOR type=write size=2846393 2024-03-04T17:50:26.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.292+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~0KR65OXKN8ZM.0 size=1048576 2024-03-04T17:50:26.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.296+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~0KR65OXKN8ZM.1 size=1048576 2024-03-04T17:50:26.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.296+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~0KR65OXKN8ZM.2 size=700093 2024-03-04T17:50:26.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.296+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=0KR65OXKN8ZM tag=PRGEY03UEWGZ type=write size=3845821 2024-03-04T17:50:26.302 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.300+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=HJUA74JW8JUN type=del 2024-03-04T17:50:26.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.304+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=MCEBXPEHS3XV type=del size=2961720 2024-03-04T17:50:26.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.308+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=YK2T3R1U3DBB type=del 2024-03-04T17:50:26.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.312+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=S3HTH7ZJV0TL type=del size=1799768 2024-03-04T17:50:26.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.316+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~F3G9PPHKCLVW.0 size=136797 2024-03-04T17:50:26.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.320+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=F3G9PPHKCLVW tag=1E1DX3MSH8JE type=write size=1185373 2024-03-04T17:50:26.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.324+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=TNU1Z08V7021 type=write size=666364 2024-03-04T17:50:26.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.328+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~VRXT1GL3RZBP.0 size=1048576 2024-03-04T17:50:26.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.332+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~VRXT1GL3RZBP.1 size=578784 2024-03-04T17:50:26.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.336+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=VRXT1GL3RZBP tag=LAMVDX68L8H6 type=write size=2675936 2024-03-04T17:50:26.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.340+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=8HE4RKVTY1Z7 type=del size=217208 2024-03-04T17:50:26.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.344+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~M3LAIL1OHATM.0 size=862297 2024-03-04T17:50:26.350 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.348+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=M3LAIL1OHATM tag=E4BVJ2Q8FC26 type=write size=1910873 2024-03-04T17:50:26.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.356+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~IGPQ2XEFYJ9G.0 size=1048576 2024-03-04T17:50:26.359 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.356+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~IGPQ2XEFYJ9G.1 size=251350 2024-03-04T17:50:26.359 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.356+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=IGPQ2XEFYJ9G tag=SW45FNUD2OTE type=write size=2348502 2024-03-04T17:50:26.363 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.360+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~VG26DBHNHKFP.0 size=1048576 2024-03-04T17:50:26.365 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.364+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~VG26DBHNHKFP.1 size=740211 2024-03-04T17:50:26.371 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.372+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=VG26DBHNHKFP tag=46EXVDQIGGBL type=write size=2837363 2024-03-04T17:50:26.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.376+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~M8SM6KGG861P.0 size=1048576 2024-03-04T17:50:26.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.380+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~M8SM6KGG861P.1 size=1048576 2024-03-04T17:50:26.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.384+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~M8SM6KGG861P.2 size=869492 2024-03-04T17:50:26.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.384+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=M8SM6KGG861P tag=GYECH0467TT1 type=write size=4015220 2024-03-04T17:50:26.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.388+0000 7fe960f568c0 1 < completed write operation key=8PZD01OOX3AF tag=0RY8JAJTAIQA type=write size=316119 2024-03-04T17:50:26.394 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.392+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~MN66OPACHHXA.0 size=780288 2024-03-04T17:50:26.394 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.392+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=MN66OPACHHXA tag=4953NQ7WS7D8 type=write size=1828864 2024-03-04T17:50:26.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.400+0000 7fe960f568c0 1 < suggested update operation key=QGLN27WH1N2O tag=KAP3X3RRC1ED type=del size=1507223 2024-03-04T17:50:26.406 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.404+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=0SH5AVZ7MKBS type=del size=1536523 2024-03-04T17:50:26.410 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.408+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=K2QP473QLJGN type=del 2024-03-04T17:50:26.414 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.412+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=4IM3Y0ND1BQY type=write size=841111 2024-03-04T17:50:26.418 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.416+0000 7fe960f568c0 1 < completed write operation key=ASC735ZCBLCG tag=YSQSNR6E77IP type=write size=173939 2024-03-04T17:50:26.425 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.424+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~EJDRUZOHG4NG.0 size=1048576 2024-03-04T17:50:26.428 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.428+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~EJDRUZOHG4NG.1 size=1048576 2024-03-04T17:50:26.431 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.432+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~EJDRUZOHG4NG.2 size=352403 2024-03-04T17:50:26.431 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.432+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=EJDRUZOHG4NG tag=WPZUIGSAI6WO type=write size=3498131 2024-03-04T17:50:26.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.436+0000 7fe960f568c0 -1 < failed to complete operation key=QGLN27WH1N2O tag=5OOZKJ6FZZU0 type=write size=17706: (22) Invalid argument 2024-03-04T17:50:26.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.440+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=7NCNUM61MQSH type=del size=62807 2024-03-04T17:50:26.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.448+0000 7fe960f568c0 1 < suggested update operation key=JP3FRB6MD7B4 tag=XH0K3K3LYCLA type=del size=1149364 2024-03-04T17:50:26.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.452+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~F65ZET38M0LU.0 size=1048576 2024-03-04T17:50:26.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.456+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~F65ZET38M0LU.1 size=790361 2024-03-04T17:50:26.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.456+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=F65ZET38M0LU tag=1M681VCVQRM6 type=write size=2887513 2024-03-04T17:50:26.462 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.460+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=7FJN0EUUPH3T type=del 2024-03-04T17:50:26.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.464+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=3VSE91GPFPK9 type=del size=1603286 2024-03-04T17:50:26.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.468+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=AZJZERIRKW4H type=del 2024-03-04T17:50:26.474 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.472+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=HPPCX659TRD7 type=write size=863277 2024-03-04T17:50:26.478 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.476+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=28HO8O4UIEOH type=del 2024-03-04T17:50:26.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.484+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~YUTC7BLHRQEJ.0 size=866027 2024-03-04T17:50:26.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.484+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=YUTC7BLHRQEJ tag=EN5XK830D7YM type=write size=1914603 2024-03-04T17:50:26.486 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.484+0000 7fe960f568c0 1 < canceled part key=_multipart_MIZRUP4MEA0J.2~0KR65OXKN8ZM.0 size=1048576 2024-03-04T17:50:26.486 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.484+0000 7fe960f568c0 1 < canceled part key=_multipart_MIZRUP4MEA0J.2~0KR65OXKN8ZM.1 size=1048576 2024-03-04T17:50:26.489 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.488+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~0KR65OXKN8ZM.2 size=700093 2024-03-04T17:50:26.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.496+0000 7fe960f568c0 1 < canceled multipart upload key=MIZRUP4MEA0J upload=0KR65OXKN8ZM tag=PRGEY03UEWGZ type=write size=3845821 2024-03-04T17:50:26.500 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.500+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=H79F9743SPBI type=write size=181468 2024-03-04T17:50:26.504 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.504+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=MCEBXPEHS3XV type=del 2024-03-04T17:50:26.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.508+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=I12GURPRYPNN type=del size=3487097 2024-03-04T17:50:26.512 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.512+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=S3HTH7ZJV0TL type=del 2024-03-04T17:50:26.516 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.516+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=5QOACMOUJN2V type=del size=2336405 2024-03-04T17:50:26.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.520+0000 7fe960f568c0 1 < completed write operation key=2FUTMG0SSXBY tag=TNU1Z08V7021 type=write size=666364 2024-03-04T17:50:26.528 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.528+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~WB0YG7EQY9EQ.0 size=1048576 2024-03-04T17:50:26.531 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.528+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~WB0YG7EQY9EQ.1 size=1048576 2024-03-04T17:50:26.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.532+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~WB0YG7EQY9EQ.2 size=333457 2024-03-04T17:50:26.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.532+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=WB0YG7EQY9EQ tag=3SQYMY2QJW93 type=write size=3479185 2024-03-04T17:50:26.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.540+0000 7fe960f568c0 1 < suggested remove operation key=P28N6U79ZUV2 tag=8HE4RKVTY1Z7 type=del 2024-03-04T17:50:26.546 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.544+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=K1TRJ9ZM1TS1 type=del size=3966337 2024-03-04T17:50:26.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.548+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~IGPQ2XEFYJ9G.0 size=1048576 2024-03-04T17:50:26.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.548+0000 7fe960f568c0 1 < canceled part key=_multipart_P28N6U79ZUV2.2~IGPQ2XEFYJ9G.1 size=251350 2024-03-04T17:50:26.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.556+0000 7fe960f568c0 1 < canceled multipart upload key=P28N6U79ZUV2 upload=IGPQ2XEFYJ9G tag=SW45FNUD2OTE type=write size=2348502 2024-03-04T17:50:26.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.560+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=CD7EXTH7ITDK type=write size=163577 2024-03-04T17:50:26.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.564+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~M8SM6KGG861P.0 size=1048576 2024-03-04T17:50:26.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.564+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~M8SM6KGG861P.1 size=1048576 2024-03-04T17:50:26.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.568+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~M8SM6KGG861P.2 size=869492 2024-03-04T17:50:26.576 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.576+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=M8SM6KGG861P tag=GYECH0467TT1 type=write size=4015220 2024-03-04T17:50:26.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.580+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=7H5ZCE12KGK8 type=del size=3598788 2024-03-04T17:50:26.588 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.588+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~MN66OPACHHXA.0 size=780288 2024-03-04T17:50:26.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.592+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=MN66OPACHHXA tag=4953NQ7WS7D8 type=write size=1828864 2024-03-04T17:50:26.601 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.600+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~4KMZCEYRNI37.0 size=77920 2024-03-04T17:50:26.601 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.600+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=4KMZCEYRNI37 tag=FU2LHC3EDPI9 type=write size=1126496 2024-03-04T17:50:26.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.608+0000 7fe960f568c0 1 < suggested update operation key=2KZ4IFW54C5B tag=0SH5AVZ7MKBS type=del size=1536523 2024-03-04T17:50:26.613 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.612+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=B4WHSA0U2HE9 type=del size=1347725 2024-03-04T17:50:26.617 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.616+0000 7fe960f568c0 1 < completed write operation key=QFF67JIW8Y8E tag=4IM3Y0ND1BQY type=write size=841111 2024-03-04T17:50:26.625 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.624+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~0LJTNLFAT1R5.0 size=1048576 2024-03-04T17:50:26.628 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.628+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~0LJTNLFAT1R5.1 size=1048576 2024-03-04T17:50:26.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.632+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~0LJTNLFAT1R5.2 size=738970 2024-03-04T17:50:26.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.632+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=0LJTNLFAT1R5 tag=8K0D9E366UXL type=write size=3884698 2024-03-04T17:50:26.635 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.636+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~EJDRUZOHG4NG.0 size=1048576 2024-03-04T17:50:26.638 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.636+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~EJDRUZOHG4NG.1 size=1048576 2024-03-04T17:50:26.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.640+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~EJDRUZOHG4NG.2 size=352403 2024-03-04T17:50:26.647 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.648+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=EJDRUZOHG4NG tag=WPZUIGSAI6WO type=write size=3498131 2024-03-04T17:50:26.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.652+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=C37AIW0DN6CV type=del size=1894661 2024-03-04T17:50:26.660 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.660+0000 7fe960f568c0 1 < suggested remove operation key=JSO6WIYMPCF6 tag=7NCNUM61MQSH type=del 2024-03-04T17:50:26.664 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.664+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=SV32PCIZT858 type=del size=3998374 2024-03-04T17:50:26.668 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.668+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~F65ZET38M0LU.0 size=1048576 2024-03-04T17:50:26.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.672+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~F65ZET38M0LU.1 size=790361 2024-03-04T17:50:26.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.676+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=F65ZET38M0LU tag=1M681VCVQRM6 type=write size=2887513 2024-03-04T17:50:26.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.684+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~K9Q456TLLCK4.0 size=40800 2024-03-04T17:50:26.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.684+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=K9Q456TLLCK4 tag=554DIKR2XAPR type=write size=1089376 2024-03-04T17:50:26.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.688+0000 7fe960f568c0 -1 < failed to complete operation key=P28N6U79ZUV2 tag=3VSE91GPFPK9 type=del size=1603286: (22) Invalid argument 2024-03-04T17:50:26.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.692+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~YYY0DK1FFVBI.0 size=1048576 2024-03-04T17:50:26.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.696+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~YYY0DK1FFVBI.1 size=337869 2024-03-04T17:50:26.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.696+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=YYY0DK1FFVBI tag=1FMEQ6SC119P type=write size=2435021 2024-03-04T17:50:26.701 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.700+0000 7fe960f568c0 1 < completed write operation key=WNXVI6K5MU3R tag=HPPCX659TRD7 type=write size=863277 2024-03-04T17:50:26.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.704+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=E2F4SG8WS71O type=del size=3921096 2024-03-04T17:50:26.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.708+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~YUTC7BLHRQEJ.0 size=866027 2024-03-04T17:50:26.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.712+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=YUTC7BLHRQEJ tag=EN5XK830D7YM type=write size=1914603 2024-03-04T17:50:26.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.716+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=R5312TIBS78H type=del size=2625278 2024-03-04T17:50:26.726 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.724+0000 7fe960f568c0 1 < suggested remove operation key=7Z7BPZ8XGBHZ tag=H79F9743SPBI type=write 2024-03-04T17:50:26.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.732+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~7393JCBYEU5P.0 size=1048576 2024-03-04T17:50:26.736 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.736+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~7393JCBYEU5P.1 size=890928 2024-03-04T17:50:26.736 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.736+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=7393JCBYEU5P tag=VTENMDSTB5PF type=write size=2988080 2024-03-04T17:50:26.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.744+0000 7fe960f568c0 1 < suggested update operation key=R6ZDVOUE9NXJ tag=I12GURPRYPNN type=del size=3487097 2024-03-04T17:50:26.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.748+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=7153L8MUVULL type=del size=3435328 2024-03-04T17:50:26.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.752+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=5QOACMOUJN2V type=del 2024-03-04T17:50:26.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.756+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=0LNT4IFCDDDH type=del size=546529 2024-03-04T17:50:26.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.756+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~WB0YG7EQY9EQ.0 size=1048576 2024-03-04T17:50:26.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.760+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~WB0YG7EQY9EQ.1 size=1048576 2024-03-04T17:50:26.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.760+0000 7fe960f568c0 1 < canceled part key=_multipart_WNXVI6K5MU3R.2~WB0YG7EQY9EQ.2 size=333457 2024-03-04T17:50:26.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.768+0000 7fe960f568c0 1 < canceled multipart upload key=WNXVI6K5MU3R upload=WB0YG7EQY9EQ tag=3SQYMY2QJW93 type=write size=3479185 2024-03-04T17:50:26.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.772+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=DUHPYCZL18UB type=write size=230301 2024-03-04T17:50:26.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.776+0000 7fe960f568c0 -1 < failed to complete operation key=2KZ4IFW54C5B tag=K1TRJ9ZM1TS1 type=del size=3966337: (22) Invalid argument 2024-03-04T17:50:26.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.780+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~XXJ7K2Z9JBNA.0 size=1048576 2024-03-04T17:50:26.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.784+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~XXJ7K2Z9JBNA.1 size=195092 2024-03-04T17:50:26.785 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.784+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=XXJ7K2Z9JBNA tag=EKXSXDSU2G8S type=write size=2292244 2024-03-04T17:50:26.788 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.788+0000 7fe960f568c0 1 < completed write operation key=JP3FRB6MD7B4 tag=CD7EXTH7ITDK type=write size=163577 2024-03-04T17:50:26.793 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.792+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=UKJC4RP2LOHY type=del size=1117964 2024-03-04T17:50:26.797 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.796+0000 7fe960f568c0 1 < canceled operation key=UKZ079SVZZKG tag=7H5ZCE12KGK8 type=del size=3598788 2024-03-04T17:50:26.801 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.800+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=2DYYD3JU3C88 type=del size=1838681 2024-03-04T17:50:26.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.804+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~4KMZCEYRNI37.0 size=77920 2024-03-04T17:50:26.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.812+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=4KMZCEYRNI37 tag=FU2LHC3EDPI9 type=write size=1126496 2024-03-04T17:50:26.817 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.816+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=69MPOMRHPN01 type=del size=285799 2024-03-04T17:50:26.822 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.820+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=B4WHSA0U2HE9 type=del 2024-03-04T17:50:26.827 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.824+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=YEVSZ37I2RQU type=del size=150831 2024-03-04T17:50:26.829 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.828+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~0LJTNLFAT1R5.0 size=1048576 2024-03-04T17:50:26.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.832+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~0LJTNLFAT1R5.1 size=1048576 2024-03-04T17:50:26.835 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.832+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~0LJTNLFAT1R5.2 size=738970 2024-03-04T17:50:26.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.840+0000 7fe960f568c0 1 < completed multipart upload key=ASC735ZCBLCG upload=0LJTNLFAT1R5 tag=8K0D9E366UXL type=write size=3884698 2024-03-04T17:50:26.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.848+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~WPUD6SY6AIFB.0 size=949294 2024-03-04T17:50:26.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.848+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=WPUD6SY6AIFB tag=F1WCPMBTRKTV type=write size=1997870 2024-03-04T17:50:26.852 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.852+0000 7fe960f568c0 1 < canceled operation key=JJ7SM1Y73A4C tag=C37AIW0DN6CV type=del size=1894661 2024-03-04T17:50:26.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.856+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=GK9SOHT1HCVU type=write size=910219 2024-03-04T17:50:26.863 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.860+0000 7fe960f568c0 1 < suggested remove operation key=GZ4RRIR5CC26 tag=SV32PCIZT858 type=del 2024-03-04T17:50:26.866 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.864+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=NQ4851J33ZNE type=del size=3656667 2024-03-04T17:50:26.867 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.868+0000 7fe960f568c0 1 < canceled part key=_multipart_QFF67JIW8Y8E.2~K9Q456TLLCK4.0 size=40800 2024-03-04T17:50:26.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.872+0000 7fe960f568c0 1 < canceled multipart upload key=QFF67JIW8Y8E upload=K9Q456TLLCK4 tag=554DIKR2XAPR type=write size=1089376 2024-03-04T17:50:26.876 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.876+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=7M5GA5ZQ3E1K type=del size=55745 2024-03-04T17:50:26.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.880+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~YYY0DK1FFVBI.0 size=1048576 2024-03-04T17:50:26.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.880+0000 7fe960f568c0 1 < canceled part key=_multipart_QFF67JIW8Y8E.2~YYY0DK1FFVBI.1 size=337869 2024-03-04T17:50:26.886 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.884+0000 7fe960f568c0 1 < canceled multipart upload key=QFF67JIW8Y8E upload=YYY0DK1FFVBI tag=1FMEQ6SC119P type=write size=2435021 2024-03-04T17:50:26.893 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.892+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~VLFXM8OBDG1W.0 size=1029361 2024-03-04T17:50:26.893 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.892+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=VLFXM8OBDG1W tag=9DH7KTOCO4TO type=write size=2077937 2024-03-04T17:50:26.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.900+0000 7fe960f568c0 1 < suggested remove operation key=QGLN27WH1N2O tag=E2F4SG8WS71O type=del 2024-03-04T17:50:26.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.908+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~9RW3YRFRSIZO.0 size=1048576 2024-03-04T17:50:26.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.908+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~9RW3YRFRSIZO.1 size=1048576 2024-03-04T17:50:26.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.912+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~9RW3YRFRSIZO.2 size=1027723 2024-03-04T17:50:26.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.912+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=9RW3YRFRSIZO tag=2YOWVGPOERUC type=write size=4173451 2024-03-04T17:50:26.916 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.916+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=R5312TIBS78H type=del 2024-03-04T17:50:26.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.916+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=OQNQRWXI0N01 type=del size=1188877 2024-03-04T17:50:26.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.920+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~7393JCBYEU5P.0 size=1048576 2024-03-04T17:50:26.926 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.924+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~7393JCBYEU5P.1 size=890928 2024-03-04T17:50:26.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.932+0000 7fe960f568c0 1 < completed multipart upload key=XSQ3GE2UHMZG upload=7393JCBYEU5P tag=VTENMDSTB5PF type=write size=2988080 2024-03-04T17:50:26.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.940+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~WY0TPJ8ONDK9.0 size=690765 2024-03-04T17:50:26.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.940+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=WY0TPJ8ONDK9 tag=C4LN7QRDLWCL type=write size=1739341 2024-03-04T17:50:26.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.948+0000 7fe960f568c0 1 < suggested update operation key=3X4719GHK3BI tag=7153L8MUVULL type=del size=3435328 2024-03-04T17:50:26.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.952+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=GYO0UHIMSVDR type=write size=255933 2024-03-04T17:50:26.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.956+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=0LNT4IFCDDDH type=del 2024-03-04T17:50:26.965 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.964+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~Y404WS2BAB8D.0 size=8035 2024-03-04T17:50:26.965 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.964+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=Y404WS2BAB8D tag=JNJ8NQGKNK3H type=write size=1056611 2024-03-04T17:50:26.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.968+0000 7fe960f568c0 1 < completed write operation key=P28N6U79ZUV2 tag=DUHPYCZL18UB type=write size=230301 2024-03-04T17:50:26.972 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.972+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=EDKUE8NRSQHR type=write size=573143 2024-03-04T17:50:26.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.976+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~XXJ7K2Z9JBNA.0 size=1048576 2024-03-04T17:50:26.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.980+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~XXJ7K2Z9JBNA.1 size=195092 2024-03-04T17:50:26.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.984+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=XXJ7K2Z9JBNA tag=EKXSXDSU2G8S type=write size=2292244 2024-03-04T17:50:26.989 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.988+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=0AYH1DP5WXPZ type=del size=3195970 2024-03-04T17:50:26.993 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:26.992+0000 7fe960f568c0 -1 < failed to complete operation key=QGLN27WH1N2O tag=UKJC4RP2LOHY type=del size=1117964: (22) Invalid argument 2024-03-04T17:50:27.000 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.000+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~3060434MNGA1.0 size=1048576 2024-03-04T17:50:27.003 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.000+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~3060434MNGA1.1 size=1048576 2024-03-04T17:50:27.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.004+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~3060434MNGA1.2 size=45331 2024-03-04T17:50:27.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.004+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=3060434MNGA1 tag=55CYGG3B3AN9 type=write size=3191059 2024-03-04T17:50:27.010 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.008+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=2DYYD3JU3C88 type=del 2024-03-04T17:50:27.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.012+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=BURCQSF383DT type=del size=2324740 2024-03-04T17:50:27.022 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.020+0000 7fe960f568c0 1 < suggested remove operation key=U2G6YEVD5506 tag=69MPOMRHPN01 type=del 2024-03-04T17:50:27.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.028+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~ZA9KOKWVVNGP.0 size=1048576 2024-03-04T17:50:27.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.032+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~ZA9KOKWVVNGP.1 size=656584 2024-03-04T17:50:27.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.032+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=ZA9KOKWVVNGP tag=ZQ2XCKC205SD type=write size=2753736 2024-03-04T17:50:27.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.036+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=YEVSZ37I2RQU type=del 2024-03-04T17:50:27.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.040+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=SLNOQ4EUWFE3 type=del size=705496 2024-03-04T17:50:27.044 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.044+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~WPUD6SY6AIFB.0 size=949294 2024-03-04T17:50:27.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.048+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=WPUD6SY6AIFB tag=F1WCPMBTRKTV type=write size=1997870 2024-03-04T17:50:27.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.056+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~VXUEOMREJA9S.0 size=1048576 2024-03-04T17:50:27.060 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.060+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~VXUEOMREJA9S.1 size=1048576 2024-03-04T17:50:27.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.060+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~VXUEOMREJA9S.2 size=1030537 2024-03-04T17:50:27.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.060+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=VXUEOMREJA9S tag=WIZ2NG0DMGG6 type=write size=4176265 2024-03-04T17:50:27.067 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.068+0000 7fe960f568c0 1 < canceled operation key=UEVMIPQ9FR81 tag=GK9SOHT1HCVU type=write size=910219 2024-03-04T17:50:27.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.072+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=7770MX2ESTCR type=del size=2341183 2024-03-04T17:50:27.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.076+0000 7fe960f568c0 1 < canceled operation key=P28N6U79ZUV2 tag=NQ4851J33ZNE type=del size=3656667 2024-03-04T17:50:27.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.080+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~KW6979FMRLTW.0 size=1048576 2024-03-04T17:50:27.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.084+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~KW6979FMRLTW.1 size=677021 2024-03-04T17:50:27.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.084+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=KW6979FMRLTW tag=AUSO95GGP3B6 type=write size=2774173 2024-03-04T17:50:27.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.088+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=7M5GA5ZQ3E1K type=del 2024-03-04T17:50:27.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.092+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=ZDF6A0EVE9HK type=del size=190960 2024-03-04T17:50:27.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.096+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~VLFXM8OBDG1W.0 size=1029361 2024-03-04T17:50:27.102 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.100+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=VLFXM8OBDG1W tag=9DH7KTOCO4TO type=write size=2077937 2024-03-04T17:50:27.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.104+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=XQN5Q7VYHL0U type=del size=1781502 2024-03-04T17:50:27.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.108+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~9RW3YRFRSIZO.0 size=1048576 2024-03-04T17:50:27.113 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.112+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~9RW3YRFRSIZO.1 size=1048576 2024-03-04T17:50:27.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.116+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~9RW3YRFRSIZO.2 size=1027723 2024-03-04T17:50:27.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.120+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=9RW3YRFRSIZO tag=2YOWVGPOERUC type=write size=4173451 2024-03-04T17:50:27.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.128+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~3MZH6AKNZR6H.0 size=1048576 2024-03-04T17:50:27.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.132+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~3MZH6AKNZR6H.1 size=174152 2024-03-04T17:50:27.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.132+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=3MZH6AKNZR6H tag=Y7W1ME0MA9RR type=write size=2271304 2024-03-04T17:50:27.136 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.136+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=OQNQRWXI0N01 type=del 2024-03-04T17:50:27.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.140+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=DXHOFI4N3CRS type=del size=3079564 2024-03-04T17:50:27.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.140+0000 7fe960f568c0 1 < canceled part key=_multipart_XSQ3GE2UHMZG.2~WY0TPJ8ONDK9.0 size=690765 2024-03-04T17:50:27.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.144+0000 7fe960f568c0 1 < canceled multipart upload key=XSQ3GE2UHMZG upload=WY0TPJ8ONDK9 tag=C4LN7QRDLWCL type=write size=1739341 2024-03-04T17:50:27.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.152+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~ZJPB2YW4C26H.0 size=140639 2024-03-04T17:50:27.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.152+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=ZJPB2YW4C26H tag=PX6KE02JZWVK type=write size=1189215 2024-03-04T17:50:27.156 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.156+0000 7fe960f568c0 1 < completed write operation key=FSWBX523X9V5 tag=GYO0UHIMSVDR type=write size=255933 2024-03-04T17:50:27.163 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.164+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~BVLM3XYK5IOU.0 size=762676 2024-03-04T17:50:27.163 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.164+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=BVLM3XYK5IOU tag=XVQSLLZPWBPH type=write size=1811252 2024-03-04T17:50:27.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.168+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~Y404WS2BAB8D.0 size=8035 2024-03-04T17:50:27.173 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.172+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=Y404WS2BAB8D tag=JNJ8NQGKNK3H type=write size=1056611 2024-03-04T17:50:27.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.176+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=PBSN0MDGCEFQ type=del size=2363199 2024-03-04T17:50:27.181 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.180+0000 7fe960f568c0 1 < canceled operation key=QGLN27WH1N2O tag=EDKUE8NRSQHR type=write size=573143 2024-03-04T17:50:27.188 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.188+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~6WLGI2ZT648E.0 size=492421 2024-03-04T17:50:27.188 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.188+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=6WLGI2ZT648E tag=R46TS3XF9AI3 type=write size=1540997 2024-03-04T17:50:27.192 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.192+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=0AYH1DP5WXPZ type=del 2024-03-04T17:50:27.196 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.196+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=UALKZ07D5D2U type=del size=2481156 2024-03-04T17:50:27.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.200+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~3060434MNGA1.0 size=1048576 2024-03-04T17:50:27.203 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.200+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~3060434MNGA1.1 size=1048576 2024-03-04T17:50:27.206 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.204+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~3060434MNGA1.2 size=45331 2024-03-04T17:50:27.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.212+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=3060434MNGA1 tag=55CYGG3B3AN9 type=write size=3191059 2024-03-04T17:50:27.218 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.216+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~5ROH49WTXOP1.0 size=12008 2024-03-04T17:50:27.218 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.216+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=5ROH49WTXOP1 tag=KGKY4P7RKODW type=write size=1060584 2024-03-04T17:50:27.222 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.220+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=BURCQSF383DT type=del 2024-03-04T17:50:27.225 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.224+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=6XMB9DHP1T3F type=del size=2060153 2024-03-04T17:50:27.230 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.228+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~ZA9KOKWVVNGP.0 size=1048576 2024-03-04T17:50:27.233 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.232+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~ZA9KOKWVVNGP.1 size=656584 2024-03-04T17:50:27.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.240+0000 7fe960f568c0 1 < completed multipart upload key=U2G6YEVD5506 upload=ZA9KOKWVVNGP tag=ZQ2XCKC205SD type=write size=2753736 2024-03-04T17:50:27.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.244+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=CRO3SJ3DCJB5 type=del size=3827988 2024-03-04T17:50:27.251 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.248+0000 7fe960f568c0 1 < suggested update operation key=FSWBX523X9V5 tag=SLNOQ4EUWFE3 type=del size=705496 2024-03-04T17:50:27.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.252+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=1L9MHFPKOADW type=del size=853918 2024-03-04T17:50:27.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.256+0000 7fe960f568c0 1 < canceled part key=_multipart_WNXVI6K5MU3R.2~VXUEOMREJA9S.0 size=1048576 2024-03-04T17:50:27.256 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.256+0000 7fe960f568c0 1 < canceled part key=_multipart_WNXVI6K5MU3R.2~VXUEOMREJA9S.1 size=1048576 2024-03-04T17:50:27.256 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.256+0000 7fe960f568c0 1 < canceled part key=_multipart_WNXVI6K5MU3R.2~VXUEOMREJA9S.2 size=1030537 2024-03-04T17:50:27.261 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.260+0000 7fe960f568c0 1 < canceled multipart upload key=WNXVI6K5MU3R upload=VXUEOMREJA9S tag=WIZ2NG0DMGG6 type=write size=4176265 2024-03-04T17:50:27.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.264+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=YFRJV3787XVO type=del size=709803 2024-03-04T17:50:27.269 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.268+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=7770MX2ESTCR type=del 2024-03-04T17:50:27.276 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.276+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~S2R526TGRP4R.0 size=440310 2024-03-04T17:50:27.276 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.276+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=S2R526TGRP4R tag=WSN6JX062ELV type=write size=1488886 2024-03-04T17:50:27.280 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.280+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~KW6979FMRLTW.0 size=1048576 2024-03-04T17:50:27.283 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.280+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~KW6979FMRLTW.1 size=677021 2024-03-04T17:50:27.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.288+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=KW6979FMRLTW tag=AUSO95GGP3B6 type=write size=2774173 2024-03-04T17:50:27.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.296+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=0DZXD5845HJI type=del size=1005063 2024-03-04T17:50:27.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.300+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=ZDF6A0EVE9HK type=del 2024-03-04T17:50:27.303 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.300+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=S7KXIQHUFT8M type=write size=11966 2024-03-04T17:50:27.307 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.304+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=XQN5Q7VYHL0U type=del 2024-03-04T17:50:27.310 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.308+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=7HPNQ6NK8X4J type=del size=1538814 2024-03-04T17:50:27.314 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.312+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~3MZH6AKNZR6H.0 size=1048576 2024-03-04T17:50:27.317 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.316+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~3MZH6AKNZR6H.1 size=174152 2024-03-04T17:50:27.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.320+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=3MZH6AKNZR6H tag=Y7W1ME0MA9RR type=write size=2271304 2024-03-04T17:50:27.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.328+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~BXRLB76NCQBS.0 size=1048576 2024-03-04T17:50:27.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.332+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~BXRLB76NCQBS.1 size=413536 2024-03-04T17:50:27.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.332+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=BXRLB76NCQBS tag=BNAGF8FPMUG7 type=write size=2510688 2024-03-04T17:50:27.336 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.336+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=DXHOFI4N3CRS type=del 2024-03-04T17:50:27.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.344+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~7SXD6YJ41EGG.0 size=1048576 2024-03-04T17:50:27.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.344+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~7SXD6YJ41EGG.1 size=1048576 2024-03-04T17:50:27.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.348+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~7SXD6YJ41EGG.2 size=1031058 2024-03-04T17:50:27.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.348+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=7SXD6YJ41EGG tag=3Q6QKOYTRL32 type=write size=4176786 2024-03-04T17:50:27.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.352+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~ZJPB2YW4C26H.0 size=140639 2024-03-04T17:50:27.358 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.356+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=ZJPB2YW4C26H tag=PX6KE02JZWVK type=write size=1189215 2024-03-04T17:50:27.365 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.364+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~NY0GNIXC369M.0 size=1048576 2024-03-04T17:50:27.367 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.368+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~NY0GNIXC369M.1 size=134945 2024-03-04T17:50:27.368 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.368+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=NY0GNIXC369M tag=9O6WQFXSY9X7 type=write size=2232097 2024-03-04T17:50:27.371 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.372+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~BVLM3XYK5IOU.0 size=762676 2024-03-04T17:50:27.377 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.376+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=BVLM3XYK5IOU tag=XVQSLLZPWBPH type=write size=1811252 2024-03-04T17:50:27.384 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.384+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~B86H82CIEL3C.0 size=1048576 2024-03-04T17:50:27.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.388+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~B86H82CIEL3C.1 size=914327 2024-03-04T17:50:27.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.388+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=B86H82CIEL3C tag=Z30NT4NHK9Q7 type=write size=3011479 2024-03-04T17:50:27.391 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.388+0000 7fe960f568c0 1 < completed delete operation key=U2G6YEVD5506 tag=PBSN0MDGCEFQ type=del 2024-03-04T17:50:27.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.396+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~N8WKYW3YMGBO.0 size=191708 2024-03-04T17:50:27.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.396+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=N8WKYW3YMGBO tag=VGAM403SUCZX type=write size=1240284 2024-03-04T17:50:27.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.400+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~6WLGI2ZT648E.0 size=492421 2024-03-04T17:50:27.407 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.408+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=6WLGI2ZT648E tag=R46TS3XF9AI3 type=write size=1540997 2024-03-04T17:50:27.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.408+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=IZNQ2XJNS118 type=write size=586735 2024-03-04T17:50:27.416 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.416+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=UALKZ07D5D2U type=del 2024-03-04T17:50:27.422 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.420+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~KZM9N96C1FWC.0 size=1048576 2024-03-04T17:50:27.425 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.424+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~KZM9N96C1FWC.1 size=753043 2024-03-04T17:50:27.425 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.424+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=KZM9N96C1FWC tag=4US3AYEBIA6W type=write size=2850195 2024-03-04T17:50:27.430 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.428+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~5ROH49WTXOP1.0 size=12008 2024-03-04T17:50:27.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.436+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=5ROH49WTXOP1 tag=KGKY4P7RKODW type=write size=1060584 2024-03-04T17:50:27.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.440+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=HTPH5ZHMDVSQ type=del size=498259 2024-03-04T17:50:27.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.440+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=6XMB9DHP1T3F type=del 2024-03-04T17:50:27.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.448+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~MLL4N70XTQ8F.0 size=1048576 2024-03-04T17:50:27.452 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.452+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~MLL4N70XTQ8F.1 size=869584 2024-03-04T17:50:27.452 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.452+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=MLL4N70XTQ8F tag=RYLPNWGFG5YS type=write size=2966736 2024-03-04T17:50:27.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.456+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=CRO3SJ3DCJB5 type=del 2024-03-04T17:50:27.459 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.460+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=XSNX6UXREKGV type=del size=2191622 2024-03-04T17:50:27.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.464+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=1L9MHFPKOADW type=del 2024-03-04T17:50:27.467 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.464+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=C4LMT3UHCOU2 type=del size=2312884 2024-03-04T17:50:27.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.468+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=YFRJV3787XVO type=del 2024-03-04T17:50:27.474 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.472+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=E2RGZQFNHKOK type=del size=1568740 2024-03-04T17:50:27.478 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.476+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~S2R526TGRP4R.0 size=440310 2024-03-04T17:50:27.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.484+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=S2R526TGRP4R tag=WSN6JX062ELV type=write size=1488886 2024-03-04T17:50:27.488 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.488+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=TE2BUI4JZRYE type=write size=755940 2024-03-04T17:50:27.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.496+0000 7fe960f568c0 1 < suggested remove operation key=ASC735ZCBLCG tag=0DZXD5845HJI type=del 2024-03-04T17:50:27.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.500+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~1AO9K0GK8PS6.0 size=1048576 2024-03-04T17:50:27.504 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.504+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~1AO9K0GK8PS6.1 size=209688 2024-03-04T17:50:27.504 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.504+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=1AO9K0GK8PS6 tag=RLXXQ5CMQVB7 type=write size=2306840 2024-03-04T17:50:27.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.508+0000 7fe960f568c0 1 < completed write operation key=QGLN27WH1N2O tag=S7KXIQHUFT8M type=write size=11966 2024-03-04T17:50:27.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.512+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=NFHX7ZFTOCKT type=del size=1287320 2024-03-04T17:50:27.515 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.512+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=7HPNQ6NK8X4J type=del 2024-03-04T17:50:27.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.520+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~6PCRERDU2LCP.0 size=1048576 2024-03-04T17:50:27.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.524+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~6PCRERDU2LCP.1 size=749249 2024-03-04T17:50:27.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.524+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=6PCRERDU2LCP tag=QOZIA5RFTSWI type=write size=2846401 2024-03-04T17:50:27.529 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.528+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~BXRLB76NCQBS.0 size=1048576 2024-03-04T17:50:27.532 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.532+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~BXRLB76NCQBS.1 size=413536 2024-03-04T17:50:27.538 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.536+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=BXRLB76NCQBS tag=BNAGF8FPMUG7 type=write size=2510688 2024-03-04T17:50:27.546 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.544+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~PWPY1R62RRXC.0 size=1048576 2024-03-04T17:50:27.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.548+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~PWPY1R62RRXC.1 size=1048576 2024-03-04T17:50:27.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.552+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~PWPY1R62RRXC.2 size=340407 2024-03-04T17:50:27.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.552+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=PWPY1R62RRXC tag=DKBVQ7K3QD3J type=write size=3486135 2024-03-04T17:50:27.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.552+0000 7fe960f568c0 1 < canceled part key=_multipart_BFF9OD00TU54.2~7SXD6YJ41EGG.0 size=1048576 2024-03-04T17:50:27.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.556+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~7SXD6YJ41EGG.1 size=1048576 2024-03-04T17:50:27.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.556+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~7SXD6YJ41EGG.2 size=1031058 2024-03-04T17:50:27.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.564+0000 7fe960f568c0 1 < canceled multipart upload key=BFF9OD00TU54 upload=7SXD6YJ41EGG tag=3Q6QKOYTRL32 type=write size=4176786 2024-03-04T17:50:27.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.568+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=SV0OIF7U89WA type=del size=1023186 2024-03-04T17:50:27.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.568+0000 7fe960f568c0 1 < canceled part key=_multipart_QGLN27WH1N2O.2~NY0GNIXC369M.0 size=1048576 2024-03-04T17:50:27.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.572+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~NY0GNIXC369M.1 size=134945 2024-03-04T17:50:27.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.576+0000 7fe960f568c0 1 < canceled multipart upload key=QGLN27WH1N2O upload=NY0GNIXC369M tag=9O6WQFXSY9X7 type=write size=2232097 2024-03-04T17:50:27.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.580+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=A0RPV1C5C5DM type=del size=3301235 2024-03-04T17:50:27.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.584+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~B86H82CIEL3C.0 size=1048576 2024-03-04T17:50:27.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.588+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~B86H82CIEL3C.1 size=914327 2024-03-04T17:50:27.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.596+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=B86H82CIEL3C tag=Z30NT4NHK9Q7 type=write size=3011479 2024-03-04T17:50:27.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.600+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~1427U0IVPMLN.0 size=1048576 2024-03-04T17:50:27.605 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.604+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~1427U0IVPMLN.1 size=1048576 2024-03-04T17:50:27.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.608+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~1427U0IVPMLN.2 size=61041 2024-03-04T17:50:27.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.608+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=1427U0IVPMLN tag=M5T9FOYTB9M9 type=write size=3206769 2024-03-04T17:50:27.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.612+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~N8WKYW3YMGBO.0 size=191708 2024-03-04T17:50:27.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.616+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=N8WKYW3YMGBO tag=VGAM403SUCZX type=write size=1240284 2024-03-04T17:50:27.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.620+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=4AFM2F2LVU3J type=del size=1931418 2024-03-04T17:50:27.630 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.628+0000 7fe960f568c0 1 < suggested remove operation key=FLYDZHOIWO6L tag=IZNQ2XJNS118 type=write 2024-03-04T17:50:27.638 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.636+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=F3OVD1PZ218Z type=del size=3080237 2024-03-04T17:50:27.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.640+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~KZM9N96C1FWC.0 size=1048576 2024-03-04T17:50:27.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.644+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~KZM9N96C1FWC.1 size=753043 2024-03-04T17:50:27.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.652+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=KZM9N96C1FWC tag=4US3AYEBIA6W type=write size=2850195 2024-03-04T17:50:27.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.656+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=4R0JMV0CR2QN type=del size=525395 2024-03-04T17:50:27.664 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.664+0000 7fe960f568c0 1 < suggested update operation key=7Z7BPZ8XGBHZ tag=HTPH5ZHMDVSQ type=del size=498259 2024-03-04T17:50:27.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.672+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~3MAZ4D0R35G5.0 size=1048576 2024-03-04T17:50:27.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.672+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~3MAZ4D0R35G5.1 size=1048576 2024-03-04T17:50:27.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.676+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~3MAZ4D0R35G5.2 size=456796 2024-03-04T17:50:27.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.676+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=3MAZ4D0R35G5 tag=PV1Y3GD5TZXL type=write size=3602524 2024-03-04T17:50:27.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.680+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~MLL4N70XTQ8F.0 size=1048576 2024-03-04T17:50:27.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.680+0000 7fe960f568c0 1 < canceled part key=_multipart_7Z7BPZ8XGBHZ.2~MLL4N70XTQ8F.1 size=869584 2024-03-04T17:50:27.686 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.684+0000 7fe960f568c0 1 < canceled multipart upload key=7Z7BPZ8XGBHZ upload=MLL4N70XTQ8F tag=RYLPNWGFG5YS type=write size=2966736 2024-03-04T17:50:27.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.688+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=0NOI5VRLNDNO type=del size=259728 2024-03-04T17:50:27.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.692+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=XSNX6UXREKGV type=del 2024-03-04T17:50:27.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.696+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=Z8OPL4FL9G93 type=write size=65085 2024-03-04T17:50:27.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.704+0000 7fe960f568c0 1 < suggested remove operation key=XSQ3GE2UHMZG tag=C4LMT3UHCOU2 type=del 2024-03-04T17:50:27.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.708+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=32JFFTJNG4N5 type=del size=433632 2024-03-04T17:50:27.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.716+0000 7fe960f568c0 1 < suggested remove operation key=JT02WL9JYTNM tag=E2RGZQFNHKOK type=del 2024-03-04T17:50:27.723 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.720+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~P80M6ZYGUA9E.0 size=1048576 2024-03-04T17:50:27.726 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.724+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~P80M6ZYGUA9E.1 size=542532 2024-03-04T17:50:27.726 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.724+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=P80M6ZYGUA9E tag=XK6TXTCJKV9M type=write size=2639684 2024-03-04T17:50:27.730 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.728+0000 7fe960f568c0 1 < completed write operation key=P28N6U79ZUV2 tag=TE2BUI4JZRYE type=write size=755940 2024-03-04T17:50:27.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.732+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=8Q7SJS1U8OGH type=del size=3011518 2024-03-04T17:50:27.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.736+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~1AO9K0GK8PS6.0 size=1048576 2024-03-04T17:50:27.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.740+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~1AO9K0GK8PS6.1 size=209688 2024-03-04T17:50:27.749 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.748+0000 7fe960f568c0 1 < completed multipart upload key=ASC735ZCBLCG upload=1AO9K0GK8PS6 tag=RLXXQ5CMQVB7 type=write size=2306840 2024-03-04T17:50:27.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.752+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=KO12TDTX0OB2 type=del size=2915197 2024-03-04T17:50:27.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.756+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=NFHX7ZFTOCKT type=del 2024-03-04T17:50:27.760 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.760+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=NKGLQ0ITJCM2 type=write size=434873 2024-03-04T17:50:27.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.764+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~6PCRERDU2LCP.0 size=1048576 2024-03-04T17:50:27.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.764+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~6PCRERDU2LCP.1 size=749249 2024-03-04T17:50:27.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.772+0000 7fe960f568c0 1 < completed multipart upload key=2MKQA5JYWU8L upload=6PCRERDU2LCP tag=QOZIA5RFTSWI type=write size=2846401 2024-03-04T17:50:27.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.776+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=86IV8WCX2OFJ type=del size=2769748 2024-03-04T17:50:27.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.780+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~PWPY1R62RRXC.0 size=1048576 2024-03-04T17:50:27.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.784+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~PWPY1R62RRXC.1 size=1048576 2024-03-04T17:50:27.789 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.788+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~PWPY1R62RRXC.2 size=340407 2024-03-04T17:50:27.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.796+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=PWPY1R62RRXC tag=DKBVQ7K3QD3J type=write size=3486135 2024-03-04T17:50:27.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.800+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=LCI7RO1QGRQM type=del size=1520068 2024-03-04T17:50:27.809 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.808+0000 7fe960f568c0 1 < suggested update operation key=JJ7SM1Y73A4C tag=SV0OIF7U89WA type=del size=1023186 2024-03-04T17:50:27.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.812+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=8IIIG5MYLHB3 type=del size=1273481 2024-03-04T17:50:27.816 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.816+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=A0RPV1C5C5DM type=del 2024-03-04T17:50:27.820 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.820+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=KZUPU92TUY6Y type=del size=2678229 2024-03-04T17:50:27.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.824+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~1427U0IVPMLN.0 size=1048576 2024-03-04T17:50:27.827 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.828+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~1427U0IVPMLN.1 size=1048576 2024-03-04T17:50:27.831 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.828+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~1427U0IVPMLN.2 size=61041 2024-03-04T17:50:27.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.836+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=1427U0IVPMLN tag=M5T9FOYTB9M9 type=write size=3206769 2024-03-04T17:50:27.842 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.840+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=X96PSPHL3NBN type=del size=1055887 2024-03-04T17:50:27.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.848+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=4AFM2F2LVU3J type=del 2024-03-04T17:50:27.855 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.852+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~FNOP3B8JO8M8.0 size=1048576 2024-03-04T17:50:27.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.856+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~FNOP3B8JO8M8.1 size=86509 2024-03-04T17:50:27.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.856+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=FNOP3B8JO8M8 tag=3I7LNSU66X8A type=write size=2183661 2024-03-04T17:50:27.863 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.860+0000 7fe960f568c0 1 < canceled operation key=NF4OIJWJ83DE tag=F3OVD1PZ218Z type=del size=3080237 2024-03-04T17:50:27.871 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.868+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~W97A3BFT2WLE.0 size=854947 2024-03-04T17:50:27.871 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.868+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=W97A3BFT2WLE tag=EEE77NSCXHR0 type=write size=1903523 2024-03-04T17:50:27.875 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.872+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=4R0JMV0CR2QN type=del 2024-03-04T17:50:27.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.880+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~CD5H29IUUIZG.0 size=18771 2024-03-04T17:50:27.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.880+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=CD5H29IUUIZG tag=5J74Y23JVJS6 type=write size=1067347 2024-03-04T17:50:27.885 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.884+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~3MAZ4D0R35G5.0 size=1048576 2024-03-04T17:50:27.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.888+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~3MAZ4D0R35G5.1 size=1048576 2024-03-04T17:50:27.891 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.892+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~3MAZ4D0R35G5.2 size=456796 2024-03-04T17:50:27.898 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.896+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=3MAZ4D0R35G5 tag=PV1Y3GD5TZXL type=write size=3602524 2024-03-04T17:50:27.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.900+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=7FMLSOSEB5GH type=del size=1269717 2024-03-04T17:50:27.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.904+0000 7fe960f568c0 1 < completed delete operation key=U2G6YEVD5506 tag=0NOI5VRLNDNO type=del 2024-03-04T17:50:27.908 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.908+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=QC9I5IAC43L3 type=write size=170524 2024-03-04T17:50:27.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.912+0000 7fe960f568c0 1 < completed write operation key=3X4719GHK3BI tag=Z8OPL4FL9G93 type=write size=65085 2024-03-04T17:50:27.916 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.916+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=KIE3GBWPYJ8J type=write size=372484 2024-03-04T17:50:27.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.920+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=32JFFTJNG4N5 type=del 2024-03-04T17:50:27.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.920+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=3FC6UUNODSFT type=del size=1306643 2024-03-04T17:50:27.927 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.924+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~P80M6ZYGUA9E.0 size=1048576 2024-03-04T17:50:27.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.928+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~P80M6ZYGUA9E.1 size=542532 2024-03-04T17:50:27.936 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.936+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=P80M6ZYGUA9E tag=XK6TXTCJKV9M type=write size=2639684 2024-03-04T17:50:27.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.940+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~B32IST5IPZFX.0 size=1048576 2024-03-04T17:50:27.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.944+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~B32IST5IPZFX.1 size=452899 2024-03-04T17:50:27.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.944+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=B32IST5IPZFX tag=2MEMYSRIMDT8 type=write size=2550051 2024-03-04T17:50:27.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.948+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=8Q7SJS1U8OGH type=del 2024-03-04T17:50:27.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.956+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~FTBHU16C3HO7.0 size=46858 2024-03-04T17:50:27.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.956+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=FTBHU16C3HO7 tag=3PWAPB6QVRQP type=write size=1095434 2024-03-04T17:50:27.960 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.960+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=KO12TDTX0OB2 type=del 2024-03-04T17:50:27.964 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.964+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=GDQQFSKCVU48 type=del size=173910 2024-03-04T17:50:27.967 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.968+0000 7fe960f568c0 1 < completed write operation key=BFF9OD00TU54 tag=NKGLQ0ITJCM2 type=write size=434873 2024-03-04T17:50:27.971 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.968+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=FBRQ2N4WAK7V type=del size=2743671 2024-03-04T17:50:27.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.972+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=86IV8WCX2OFJ type=del 2024-03-04T17:50:27.978 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.976+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=FXB8C6EKD4U7 type=del size=3587191 2024-03-04T17:50:27.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.984+0000 7fe960f568c0 1 < suggested update operation key=QGLN27WH1N2O tag=LCI7RO1QGRQM type=del size=1520068 2024-03-04T17:50:27.993 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.992+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~0WMDWVPCX4WA.0 size=1048576 2024-03-04T17:50:27.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:27.996+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~0WMDWVPCX4WA.1 size=1048576 2024-03-04T17:50:28.004 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.004+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~0WMDWVPCX4WA.2 size=516570 2024-03-04T17:50:28.004 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.004+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=0WMDWVPCX4WA tag=F1XJPFZT1VI3 type=write size=3662298 2024-03-04T17:50:28.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.008+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=8IIIG5MYLHB3 type=del 2024-03-04T17:50:28.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.012+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~Y64W9CXQRL4O.0 size=1048576 2024-03-04T17:50:28.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.016+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~Y64W9CXQRL4O.1 size=1048576 2024-03-04T17:50:28.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.020+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~Y64W9CXQRL4O.2 size=304940 2024-03-04T17:50:28.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.020+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=Y64W9CXQRL4O tag=S86HSNCLPL7U type=write size=3450668 2024-03-04T17:50:28.027 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.024+0000 7fe960f568c0 1 < suggested update operation key=JJ7SM1Y73A4C tag=KZUPU92TUY6Y type=del size=2678229 2024-03-04T17:50:28.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.028+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=0RWMFISUXCSC type=del size=2320677 2024-03-04T17:50:28.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.032+0000 7fe960f568c0 -1 < failed to complete operation key=QGLN27WH1N2O tag=X96PSPHL3NBN type=del size=1055887: (22) Invalid argument 2024-03-04T17:50:28.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.036+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=7Q44XKVQ2ODZ type=write size=19599 2024-03-04T17:50:28.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.040+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~FNOP3B8JO8M8.0 size=1048576 2024-03-04T17:50:28.044 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.044+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~FNOP3B8JO8M8.1 size=86509 2024-03-04T17:50:28.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.048+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=FNOP3B8JO8M8 tag=3I7LNSU66X8A type=write size=2183661 2024-03-04T17:50:28.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.056+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~VFQD5380Z2DV.0 size=584838 2024-03-04T17:50:28.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.056+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=VFQD5380Z2DV tag=8VSPVXC6I0SC type=write size=1633414 2024-03-04T17:50:28.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.060+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~W97A3BFT2WLE.0 size=854947 2024-03-04T17:50:28.067 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.064+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=W97A3BFT2WLE tag=EEE77NSCXHR0 type=write size=1903523 2024-03-04T17:50:28.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.072+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~3BT33Q7EYAVO.0 size=292236 2024-03-04T17:50:28.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.072+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=3BT33Q7EYAVO tag=QYD7XZE7Q658 type=write size=1340812 2024-03-04T17:50:28.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.076+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~CD5H29IUUIZG.0 size=18771 2024-03-04T17:50:28.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.084+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=CD5H29IUUIZG tag=5J74Y23JVJS6 type=write size=1067347 2024-03-04T17:50:28.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.088+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=S59N9EDM7EM3 type=del size=2088165 2024-03-04T17:50:28.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.096+0000 7fe960f568c0 1 < suggested update operation key=MIZRUP4MEA0J tag=7FMLSOSEB5GH type=del size=1269717 2024-03-04T17:50:28.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.100+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=BPSHT6HTBQWL type=del size=1601038 2024-03-04T17:50:28.104 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.104+0000 7fe960f568c0 1 < completed write operation key=78N8HHC4O4CG tag=QC9I5IAC43L3 type=write size=170524 2024-03-04T17:50:28.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.108+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=DP3DZFP25KSS type=del size=2945847 2024-03-04T17:50:28.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.116+0000 7fe960f568c0 1 < suggested remove operation key=ASC735ZCBLCG tag=KIE3GBWPYJ8J type=write 2024-03-04T17:50:28.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.120+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=Y5C1G9QUH8ZF type=del size=1376499 2024-03-04T17:50:28.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.124+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=3FC6UUNODSFT type=del 2024-03-04T17:50:28.128 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.128+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=0FN9WYH5Q5EY type=del size=2853753 2024-03-04T17:50:28.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.132+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~B32IST5IPZFX.0 size=1048576 2024-03-04T17:50:28.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.132+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~B32IST5IPZFX.1 size=452899 2024-03-04T17:50:28.141 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.140+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=B32IST5IPZFX tag=2MEMYSRIMDT8 type=write size=2550051 2024-03-04T17:50:28.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.144+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=CR6NS22NAXEA type=del size=2639621 2024-03-04T17:50:28.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.148+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~FTBHU16C3HO7.0 size=46858 2024-03-04T17:50:28.153 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.152+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=FTBHU16C3HO7 tag=3PWAPB6QVRQP type=write size=1095434 2024-03-04T17:50:28.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.160+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~H2GSXY90AZ0B.0 size=260949 2024-03-04T17:50:28.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.160+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=H2GSXY90AZ0B tag=CFN2P4ENFIR4 type=write size=1309525 2024-03-04T17:50:28.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.168+0000 7fe960f568c0 1 < suggested update operation key=NF4OIJWJ83DE tag=GDQQFSKCVU48 type=del size=173910 2024-03-04T17:50:28.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.172+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~2UHJATGJIAIK.0 size=794634 2024-03-04T17:50:28.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.172+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=2UHJATGJIAIK tag=M57CRYD3YHWD type=write size=1843210 2024-03-04T17:50:28.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.176+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=FBRQ2N4WAK7V type=del 2024-03-04T17:50:28.182 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.180+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=H1GGFTIC6YKQ type=del size=3986237 2024-03-04T17:50:28.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.184+0000 7fe960f568c0 1 < canceled operation key=RII14ZGX2EL1 tag=FXB8C6EKD4U7 type=del size=3587191 2024-03-04T17:50:28.189 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.188+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=LJ4GFHPNXOBU type=write size=513198 2024-03-04T17:50:28.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.192+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~0WMDWVPCX4WA.0 size=1048576 2024-03-04T17:50:28.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.196+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~0WMDWVPCX4WA.1 size=1048576 2024-03-04T17:50:28.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.196+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~0WMDWVPCX4WA.2 size=516570 2024-03-04T17:50:28.204 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.204+0000 7fe960f568c0 1 < completed multipart upload key=2MKQA5JYWU8L upload=0WMDWVPCX4WA tag=F1XJPFZT1VI3 type=write size=3662298 2024-03-04T17:50:28.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.208+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~4JKE89QGXS3K.0 size=1048576 2024-03-04T17:50:28.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.212+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~4JKE89QGXS3K.1 size=1048576 2024-03-04T17:50:28.216 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.216+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~4JKE89QGXS3K.2 size=331704 2024-03-04T17:50:28.216 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.216+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=4JKE89QGXS3K tag=K4X62L9MAUX2 type=write size=3477432 2024-03-04T17:50:28.221 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.220+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~Y64W9CXQRL4O.0 size=1048576 2024-03-04T17:50:28.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.224+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~Y64W9CXQRL4O.1 size=1048576 2024-03-04T17:50:28.227 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.224+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~Y64W9CXQRL4O.2 size=304940 2024-03-04T17:50:28.233 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.232+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=Y64W9CXQRL4O tag=S86HSNCLPL7U type=write size=3450668 2024-03-04T17:50:28.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.236+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=IXDR35HR4VZP type=del size=3830082 2024-03-04T17:50:28.240 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.240+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=0RWMFISUXCSC type=del 2024-03-04T17:50:28.244 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.244+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=JVDGLDITU1JW type=del size=3518405 2024-03-04T17:50:28.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.248+0000 7fe960f568c0 -1 < failed to complete operation key=ASC735ZCBLCG tag=7Q44XKVQ2ODZ type=write size=19599: (22) Invalid argument 2024-03-04T17:50:28.252 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.252+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=YP6KE190T3Q9 type=del size=2079337 2024-03-04T17:50:28.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.256+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~VFQD5380Z2DV.0 size=584838 2024-03-04T17:50:28.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.260+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=VFQD5380Z2DV tag=8VSPVXC6I0SC type=write size=1633414 2024-03-04T17:50:28.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.264+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=7BC92CETTSNL type=del size=4064381 2024-03-04T17:50:28.271 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.268+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~3BT33Q7EYAVO.0 size=292236 2024-03-04T17:50:28.276 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.276+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=3BT33Q7EYAVO tag=QYD7XZE7Q658 type=write size=1340812 2024-03-04T17:50:28.283 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.280+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~CFD2C5WH912F.0 size=1048576 2024-03-04T17:50:28.285 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.284+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~CFD2C5WH912F.1 size=1048576 2024-03-04T17:50:28.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.288+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~CFD2C5WH912F.2 size=305667 2024-03-04T17:50:28.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.288+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=CFD2C5WH912F tag=PT3VYLWMAM4X type=write size=3451395 2024-03-04T17:50:28.292 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.292+0000 7fe960f568c0 1 < canceled operation key=XSQ3GE2UHMZG tag=S59N9EDM7EM3 type=del size=2088165 2024-03-04T17:50:28.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.296+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=1CENMNOEW02H type=del size=798 2024-03-04T17:50:28.302 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.300+0000 7fe960f568c0 1 < suggested update operation key=78N8HHC4O4CG tag=BPSHT6HTBQWL type=del size=1601038 2024-03-04T17:50:28.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.308+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~XFWODE6CCCEZ.0 size=293909 2024-03-04T17:50:28.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.308+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=XFWODE6CCCEZ tag=ZGHMYY48FDKA type=write size=1342485 2024-03-04T17:50:28.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.312+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=DP3DZFP25KSS type=del 2024-03-04T17:50:28.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.316+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=L4MTJ01RY2OQ type=del size=2963684 2024-03-04T17:50:28.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.320+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=Y5C1G9QUH8ZF type=del 2024-03-04T17:50:28.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.320+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=1C68NQIMK4HA type=del size=2435252 2024-03-04T17:50:28.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.328+0000 7fe960f568c0 1 < suggested remove operation key=XSQ3GE2UHMZG tag=0FN9WYH5Q5EY type=del 2024-03-04T17:50:28.336 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.336+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~AODQ6L4J2X81.0 size=604559 2024-03-04T17:50:28.336 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.336+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=AODQ6L4J2X81 tag=80XUFLWWWRBP type=write size=1653135 2024-03-04T17:50:28.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.340+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=CR6NS22NAXEA type=del 2024-03-04T17:50:28.351 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.348+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~WKEXEHC4ST86.0 size=1048576 2024-03-04T17:50:28.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.352+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~WKEXEHC4ST86.1 size=973731 2024-03-04T17:50:28.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.352+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=WKEXEHC4ST86 tag=ZRVIC3ONTNU2 type=write size=3070883 2024-03-04T17:50:28.358 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.356+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~H2GSXY90AZ0B.0 size=260949 2024-03-04T17:50:28.364 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.364+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=H2GSXY90AZ0B tag=CFN2P4ENFIR4 type=write size=1309525 2024-03-04T17:50:28.368 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.368+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=HKBS9TYMO9Y1 type=del size=2419845 2024-03-04T17:50:28.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.372+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~2UHJATGJIAIK.0 size=794634 2024-03-04T17:50:28.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.376+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=2UHJATGJIAIK tag=M57CRYD3YHWD type=write size=1843210 2024-03-04T17:50:28.384 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.384+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~QLX2HUWAFZRT.0 size=446602 2024-03-04T17:50:28.384 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.384+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=QLX2HUWAFZRT tag=VGSMFLLQAFEU type=write size=1495178 2024-03-04T17:50:28.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.388+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=H1GGFTIC6YKQ type=del 2024-03-04T17:50:28.391 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.392+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=IVUOPUOHVBMS type=write size=844422 2024-03-04T17:50:28.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.392+0000 7fe960f568c0 1 < canceled operation key=UKZ079SVZZKG tag=LJ4GFHPNXOBU type=write size=513198 2024-03-04T17:50:28.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.400+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~T9C4M7DASNHQ.0 size=1048576 2024-03-04T17:50:28.404 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.404+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~T9C4M7DASNHQ.1 size=138339 2024-03-04T17:50:28.404 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.404+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=T9C4M7DASNHQ tag=0PXEFZ3MD08W type=write size=2235491 2024-03-04T17:50:28.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.408+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~4JKE89QGXS3K.0 size=1048576 2024-03-04T17:50:28.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.408+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~4JKE89QGXS3K.1 size=1048576 2024-03-04T17:50:28.413 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.412+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~4JKE89QGXS3K.2 size=331704 2024-03-04T17:50:28.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.420+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=4JKE89QGXS3K tag=K4X62L9MAUX2 type=write size=3477432 2024-03-04T17:50:28.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.424+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~SEJM1EXRAVOA.0 size=1048576 2024-03-04T17:50:28.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.428+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~SEJM1EXRAVOA.1 size=729075 2024-03-04T17:50:28.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.428+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=SEJM1EXRAVOA tag=IAM57AG8S100 type=write size=2826227 2024-03-04T17:50:28.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.432+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=IXDR35HR4VZP type=del 2024-03-04T17:50:28.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.440+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~U9QAN8QMIFQF.0 size=1048576 2024-03-04T17:50:28.442 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.440+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~U9QAN8QMIFQF.1 size=159846 2024-03-04T17:50:28.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.440+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=U9QAN8QMIFQF tag=CEPLYDYYCS6R type=write size=2256998 2024-03-04T17:50:28.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.444+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=JVDGLDITU1JW type=del 2024-03-04T17:50:28.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.448+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=PSJTW009GNOI type=del size=3622850 2024-03-04T17:50:28.454 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.452+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=YP6KE190T3Q9 type=del 2024-03-04T17:50:28.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.456+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=467N9S07E9CK type=del size=4181764 2024-03-04T17:50:28.462 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.460+0000 7fe960f568c0 1 < canceled operation key=GZ4RRIR5CC26 tag=7BC92CETTSNL type=del size=4064381 2024-03-04T17:50:28.467 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.464+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=D7MLL2F98220 type=del size=1382438 2024-03-04T17:50:28.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.468+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~CFD2C5WH912F.0 size=1048576 2024-03-04T17:50:28.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.472+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~CFD2C5WH912F.1 size=1048576 2024-03-04T17:50:28.476 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.476+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~CFD2C5WH912F.2 size=305667 2024-03-04T17:50:28.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.480+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=CFD2C5WH912F tag=PT3VYLWMAM4X type=write size=3451395 2024-03-04T17:50:28.487 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.484+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=Z342C5MIG9H6 type=del size=3205771 2024-03-04T17:50:28.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.492+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=1CENMNOEW02H type=del 2024-03-04T17:50:28.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.496+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=YJ5C1Q9EEYSZ type=write size=901718 2024-03-04T17:50:28.500 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.500+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~XFWODE6CCCEZ.0 size=293909 2024-03-04T17:50:28.507 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.504+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=XFWODE6CCCEZ tag=ZGHMYY48FDKA type=write size=1342485 2024-03-04T17:50:28.514 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.512+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~1H9QGUOIYZY2.0 size=740277 2024-03-04T17:50:28.514 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.512+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=1H9QGUOIYZY2 tag=REOPTD6BBPQ2 type=write size=1788853 2024-03-04T17:50:28.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.520+0000 7fe960f568c0 1 < suggested update operation key=QFF67JIW8Y8E tag=L4MTJ01RY2OQ type=del size=2963684 2024-03-04T17:50:28.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.524+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=86V4X5XQBELQ type=del size=2475742 2024-03-04T17:50:28.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.532+0000 7fe960f568c0 1 < suggested update operation key=QGLN27WH1N2O tag=1C68NQIMK4HA type=del size=2435252 2024-03-04T17:50:28.537 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.536+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=IANEA8KCKHTE type=del size=823817 2024-03-04T17:50:28.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.540+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~AODQ6L4J2X81.0 size=604559 2024-03-04T17:50:28.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.544+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=AODQ6L4J2X81 tag=80XUFLWWWRBP type=write size=1653135 2024-03-04T17:50:28.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.548+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=M4G0QM6ZI255 type=del size=344190 2024-03-04T17:50:28.554 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.552+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~WKEXEHC4ST86.0 size=1048576 2024-03-04T17:50:28.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.556+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~WKEXEHC4ST86.1 size=973731 2024-03-04T17:50:28.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.560+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=WKEXEHC4ST86 tag=ZRVIC3ONTNU2 type=write size=3070883 2024-03-04T17:50:28.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.568+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=RFW3FBYUU37T type=del size=546928 2024-03-04T17:50:28.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.572+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=HKBS9TYMO9Y1 type=del 2024-03-04T17:50:28.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.576+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~J9MY202412P8.0 size=1048576 2024-03-04T17:50:28.581 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.580+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~J9MY202412P8.1 size=1048576 2024-03-04T17:50:28.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.584+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~J9MY202412P8.2 size=763968 2024-03-04T17:50:28.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.584+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=J9MY202412P8 tag=0D8WZ2YEPTXK type=write size=3909696 2024-03-04T17:50:28.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.588+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~QLX2HUWAFZRT.0 size=446602 2024-03-04T17:50:28.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.592+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=QLX2HUWAFZRT tag=VGSMFLLQAFEU type=write size=1495178 2024-03-04T17:50:28.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.596+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=0FF2N4IXMV6B type=write size=518490 2024-03-04T17:50:28.601 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.600+0000 7fe960f568c0 1 < completed write operation key=R6ZDVOUE9NXJ tag=IVUOPUOHVBMS type=write size=844422 2024-03-04T17:50:28.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.608+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~2VIHN7JMI4XR.0 size=258901 2024-03-04T17:50:28.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.608+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=2VIHN7JMI4XR tag=X0T291S0AREN type=write size=1307477 2024-03-04T17:50:28.611 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.612+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~T9C4M7DASNHQ.0 size=1048576 2024-03-04T17:50:28.614 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.612+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~T9C4M7DASNHQ.1 size=138339 2024-03-04T17:50:28.621 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.620+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=T9C4M7DASNHQ tag=0PXEFZ3MD08W type=write size=2235491 2024-03-04T17:50:28.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.628+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~JH7D45GOC1TM.0 size=883409 2024-03-04T17:50:28.628 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.628+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=JH7D45GOC1TM tag=BJONEH48T07A type=write size=1931985 2024-03-04T17:50:28.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.632+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~SEJM1EXRAVOA.0 size=1048576 2024-03-04T17:50:28.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.632+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~SEJM1EXRAVOA.1 size=729075 2024-03-04T17:50:28.640 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.640+0000 7fe960f568c0 1 < completed multipart upload key=U2G6YEVD5506 upload=SEJM1EXRAVOA tag=IAM57AG8S100 type=write size=2826227 2024-03-04T17:50:28.643 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.644+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=OSO8V747VG3J type=del size=714854 2024-03-04T17:50:28.647 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.648+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~U9QAN8QMIFQF.0 size=1048576 2024-03-04T17:50:28.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.648+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~U9QAN8QMIFQF.1 size=159846 2024-03-04T17:50:28.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.656+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=U9QAN8QMIFQF tag=CEPLYDYYCS6R type=write size=2256998 2024-03-04T17:50:28.660 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.660+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=RVO347OWTVR6 type=del size=398614 2024-03-04T17:50:28.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.664+0000 7fe960f568c0 1 < suggested update operation key=2KZ4IFW54C5B tag=PSJTW009GNOI type=del size=3622850 2024-03-04T17:50:28.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.668+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=O6XEA0JJ9Q9U type=del size=1740591 2024-03-04T17:50:28.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.676+0000 7fe960f568c0 1 < suggested update operation key=RII14ZGX2EL1 tag=467N9S07E9CK type=del size=4181764 2024-03-04T17:50:28.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.684+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~3SSU3R2U09OP.0 size=1048576 2024-03-04T17:50:28.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.684+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~3SSU3R2U09OP.1 size=141726 2024-03-04T17:50:28.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.684+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=3SSU3R2U09OP tag=4DWTZMA6YO1O type=write size=2238878 2024-03-04T17:50:28.691 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.688+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=D7MLL2F98220 type=del 2024-03-04T17:50:28.701 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.700+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~2RZLABPLLXLQ.0 size=1048576 2024-03-04T17:50:28.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.704+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~2RZLABPLLXLQ.1 size=1048576 2024-03-04T17:50:28.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.704+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~2RZLABPLLXLQ.2 size=973832 2024-03-04T17:50:28.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.704+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=2RZLABPLLXLQ tag=H5IGSLOS6XA2 type=write size=4119560 2024-03-04T17:50:28.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.712+0000 7fe960f568c0 1 < suggested remove operation key=QGLN27WH1N2O tag=Z342C5MIG9H6 type=del 2024-03-04T17:50:28.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.720+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=X403E2K97ZEA type=del size=1873142 2024-03-04T17:50:28.729 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.728+0000 7fe960f568c0 1 < suggested remove operation key=78N8HHC4O4CG tag=YJ5C1Q9EEYSZ type=write 2024-03-04T17:50:28.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.732+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=SASWD21AZRB1 type=del size=1182588 2024-03-04T17:50:28.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.736+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~1H9QGUOIYZY2.0 size=740277 2024-03-04T17:50:28.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.744+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=1H9QGUOIYZY2 tag=REOPTD6BBPQ2 type=write size=1788853 2024-03-04T17:50:28.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.748+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~65Z1U1Q427Q7.0 size=1048576 2024-03-04T17:50:28.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.752+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~65Z1U1Q427Q7.1 size=1029633 2024-03-04T17:50:28.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.752+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=65Z1U1Q427Q7 tag=07NLJ2EXAY2R type=write size=3126785 2024-03-04T17:50:28.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.756+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=86V4X5XQBELQ type=del 2024-03-04T17:50:28.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.760+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=JZDIW0T5506C type=del size=2576767 2024-03-04T17:50:28.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.768+0000 7fe960f568c0 1 < suggested update operation key=BFF9OD00TU54 tag=IANEA8KCKHTE type=del size=823817 2024-03-04T17:50:28.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.772+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=UANX0ZQXINOZ type=del size=3743138 2024-03-04T17:50:28.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.776+0000 7fe960f568c0 -1 < failed to complete operation key=BFF9OD00TU54 tag=M4G0QM6ZI255 type=del size=344190: (22) Invalid argument 2024-03-04T17:50:28.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.784+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~FWA1FX57PXMK.0 size=884830 2024-03-04T17:50:28.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.784+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=FWA1FX57PXMK tag=T609U13CBOB9 type=write size=1933406 2024-03-04T17:50:28.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.788+0000 7fe960f568c0 1 < suggested remove operation key=BFF9OD00TU54 tag=RFW3FBYUU37T type=del 2024-03-04T17:50:28.797 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.796+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~H164KZ5VQWQE.0 size=1048576 2024-03-04T17:50:28.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.800+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~H164KZ5VQWQE.1 size=345245 2024-03-04T17:50:28.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.800+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=H164KZ5VQWQE tag=I2T30P9FP1JA type=write size=2442397 2024-03-04T17:50:28.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.804+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~J9MY202412P8.0 size=1048576 2024-03-04T17:50:28.808 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.808+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~J9MY202412P8.1 size=1048576 2024-03-04T17:50:28.810 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.808+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~J9MY202412P8.2 size=763968 2024-03-04T17:50:28.816 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.816+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=J9MY202412P8 tag=0D8WZ2YEPTXK type=write size=3909696 2024-03-04T17:50:28.823 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.820+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~4IRZD1SNSN5Z.0 size=1048576 2024-03-04T17:50:28.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.824+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~4IRZD1SNSN5Z.1 size=1048576 2024-03-04T17:50:28.828 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.828+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~4IRZD1SNSN5Z.2 size=604353 2024-03-04T17:50:28.828 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.828+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=4IRZD1SNSN5Z tag=O5F0J1RZ0FSK type=write size=3750081 2024-03-04T17:50:28.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.832+0000 7fe960f568c0 1 < completed write operation key=XSQ3GE2UHMZG tag=0FF2N4IXMV6B type=write size=518490 2024-03-04T17:50:28.836 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.836+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=HU9IKZA14UC5 type=del size=57192 2024-03-04T17:50:28.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.840+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~2VIHN7JMI4XR.0 size=258901 2024-03-04T17:50:28.846 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.844+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=2VIHN7JMI4XR tag=X0T291S0AREN type=write size=1307477 2024-03-04T17:50:28.852 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.852+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~3SNOO3P1VZ6N.0 size=1048576 2024-03-04T17:50:28.855 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.852+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~3SNOO3P1VZ6N.1 size=1048576 2024-03-04T17:50:28.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.856+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~3SNOO3P1VZ6N.2 size=705113 2024-03-04T17:50:28.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.856+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=3SNOO3P1VZ6N tag=2RYG1PIZOHN3 type=write size=3850841 2024-03-04T17:50:28.861 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.860+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~JH7D45GOC1TM.0 size=883409 2024-03-04T17:50:28.867 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.864+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=JH7D45GOC1TM tag=BJONEH48T07A type=write size=1931985 2024-03-04T17:50:28.874 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.872+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~QS4K3FRCIM9D.0 size=1048576 2024-03-04T17:50:28.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.876+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~QS4K3FRCIM9D.1 size=464957 2024-03-04T17:50:28.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.876+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=QS4K3FRCIM9D tag=8UTMKS0TG4IT type=write size=2562109 2024-03-04T17:50:28.880 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.880+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=OSO8V747VG3J type=del 2024-03-04T17:50:28.887 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.884+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~313DEHL1NWJ6.0 size=1048576 2024-03-04T17:50:28.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.888+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~313DEHL1NWJ6.1 size=1048576 2024-03-04T17:50:28.892 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.892+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~313DEHL1NWJ6.2 size=1008950 2024-03-04T17:50:28.892 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.892+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=313DEHL1NWJ6 tag=LCQZLMXK5CZ6 type=write size=4154678 2024-03-04T17:50:28.896 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.896+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=RVO347OWTVR6 type=del 2024-03-04T17:50:28.900 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.900+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=NJDJ8TEHY600 type=write size=968072 2024-03-04T17:50:28.904 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.904+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=O6XEA0JJ9Q9U type=del 2024-03-04T17:50:28.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.908+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=NO18G3RNM0AQ type=write size=278092 2024-03-04T17:50:28.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.908+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~3SSU3R2U09OP.0 size=1048576 2024-03-04T17:50:28.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.912+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~3SSU3R2U09OP.1 size=141726 2024-03-04T17:50:28.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.920+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=3SSU3R2U09OP tag=4DWTZMA6YO1O type=write size=2238878 2024-03-04T17:50:28.924 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.924+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=MZ4YZVV05MS4 type=del size=3174459 2024-03-04T17:50:28.928 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.928+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~2RZLABPLLXLQ.0 size=1048576 2024-03-04T17:50:28.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.932+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~2RZLABPLLXLQ.1 size=1048576 2024-03-04T17:50:28.934 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.932+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~2RZLABPLLXLQ.2 size=973832 2024-03-04T17:50:28.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.940+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=2RZLABPLLXLQ tag=H5IGSLOS6XA2 type=write size=4119560 2024-03-04T17:50:28.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.944+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=HBL5PVJMWKK8 type=del size=940955 2024-03-04T17:50:28.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.952+0000 7fe960f568c0 1 < suggested update operation key=XSQ3GE2UHMZG tag=X403E2K97ZEA type=del size=1873142 2024-03-04T17:50:28.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.956+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=H37DDRWHRHVY type=del size=948330 2024-03-04T17:50:28.960 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.960+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=SASWD21AZRB1 type=del 2024-03-04T17:50:28.964 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.964+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=3R6XHJC0CLMH type=del size=2461604 2024-03-04T17:50:28.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.968+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~65Z1U1Q427Q7.0 size=1048576 2024-03-04T17:50:28.971 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.968+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~65Z1U1Q427Q7.1 size=1029633 2024-03-04T17:50:28.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.976+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=65Z1U1Q427Q7 tag=07NLJ2EXAY2R type=write size=3126785 2024-03-04T17:50:28.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.980+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=9YSXQJ7TUE7R type=del size=2759186 2024-03-04T17:50:28.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.984+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=JZDIW0T5506C type=del 2024-03-04T17:50:28.988 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.988+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=U0443TMAGCRI type=del size=1430683 2024-03-04T17:50:28.992 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.992+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=UANX0ZQXINOZ type=del 2024-03-04T17:50:28.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:28.996+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=P5LPAP3IO4TL type=del size=1134058 2024-03-04T17:50:29.000 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.000+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~FWA1FX57PXMK.0 size=884830 2024-03-04T17:50:29.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.004+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=FWA1FX57PXMK tag=T609U13CBOB9 type=write size=1933406 2024-03-04T17:50:29.013 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.012+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~5IJL84I8LH2V.0 size=290923 2024-03-04T17:50:29.013 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.012+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=5IJL84I8LH2V tag=PU1U3L6BN3AA type=write size=1339499 2024-03-04T17:50:29.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.012+0000 7fe960f568c0 1 < canceled part key=_multipart_R6ZDVOUE9NXJ.2~H164KZ5VQWQE.0 size=1048576 2024-03-04T17:50:29.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.016+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~H164KZ5VQWQE.1 size=345245 2024-03-04T17:50:29.023 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.024+0000 7fe960f568c0 1 < canceled multipart upload key=R6ZDVOUE9NXJ upload=H164KZ5VQWQE tag=I2T30P9FP1JA type=write size=2442397 2024-03-04T17:50:29.031 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.032+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~PXBPPPVRY68J.0 size=924054 2024-03-04T17:50:29.031 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.032+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=PXBPPPVRY68J tag=MX1IWBMIUY35 type=write size=1972630 2024-03-04T17:50:29.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.032+0000 7fe960f568c0 1 < canceled part key=_multipart_768X2XKWJW1S.2~4IRZD1SNSN5Z.0 size=1048576 2024-03-04T17:50:29.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.036+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~4IRZD1SNSN5Z.1 size=1048576 2024-03-04T17:50:29.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.036+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~4IRZD1SNSN5Z.2 size=604353 2024-03-04T17:50:29.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.048+0000 7fe960f568c0 1 < canceled multipart upload key=768X2XKWJW1S upload=4IRZD1SNSN5Z tag=O5F0J1RZ0FSK type=write size=3750081 2024-03-04T17:50:29.052 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.052+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=5LDPCAFQ41UW type=del size=1392390 2024-03-04T17:50:29.056 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.060+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=HU9IKZA14UC5 type=del 2024-03-04T17:50:29.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.060+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~CGB11GSDIZYP.0 size=1048576 2024-03-04T17:50:29.066 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.064+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~CGB11GSDIZYP.1 size=462556 2024-03-04T17:50:29.066 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.064+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=CGB11GSDIZYP tag=SC6V5D22SW3O type=write size=2559708 2024-03-04T17:50:29.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.068+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~3SNOO3P1VZ6N.0 size=1048576 2024-03-04T17:50:29.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.068+0000 7fe960f568c0 1 < canceled part key=_multipart_QFF67JIW8Y8E.2~3SNOO3P1VZ6N.1 size=1048576 2024-03-04T17:50:29.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.072+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~3SNOO3P1VZ6N.2 size=705113 2024-03-04T17:50:29.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.076+0000 7fe960f568c0 1 < canceled multipart upload key=QFF67JIW8Y8E upload=3SNOO3P1VZ6N tag=2RYG1PIZOHN3 type=write size=3850841 2024-03-04T17:50:29.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.084+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~32RYHL5HMSV7.0 size=1048576 2024-03-04T17:50:29.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.088+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~32RYHL5HMSV7.1 size=1048576 2024-03-04T17:50:29.091 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.092+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~32RYHL5HMSV7.2 size=482354 2024-03-04T17:50:29.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.092+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=32RYHL5HMSV7 tag=B0BXQFWAH0NJ type=write size=3628082 2024-03-04T17:50:29.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.096+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~QS4K3FRCIM9D.0 size=1048576 2024-03-04T17:50:29.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.096+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~QS4K3FRCIM9D.1 size=464957 2024-03-04T17:50:29.105 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.104+0000 7fe960f568c0 1 < completed multipart upload key=ASC735ZCBLCG upload=QS4K3FRCIM9D tag=8UTMKS0TG4IT type=write size=2562109 2024-03-04T17:50:29.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.108+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=ZF1C7FCHEXPI type=write size=965993 2024-03-04T17:50:29.113 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.112+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~313DEHL1NWJ6.0 size=1048576 2024-03-04T17:50:29.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.116+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~313DEHL1NWJ6.1 size=1048576 2024-03-04T17:50:29.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.116+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~313DEHL1NWJ6.2 size=1008950 2024-03-04T17:50:29.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.124+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=313DEHL1NWJ6 tag=LCQZLMXK5CZ6 type=write size=4154678 2024-03-04T17:50:29.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.128+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=QXLXCNDKF04Q type=del size=1542204 2024-03-04T17:50:29.133 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.132+0000 7fe960f568c0 1 < completed write operation key=768X2XKWJW1S tag=NJDJ8TEHY600 type=write size=968072 2024-03-04T17:50:29.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.136+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=6720FO4K0YOZ type=write size=850251 2024-03-04T17:50:29.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.140+0000 7fe960f568c0 1 < completed write operation key=QFF67JIW8Y8E tag=NO18G3RNM0AQ type=write size=278092 2024-03-04T17:50:29.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.148+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~EMX8ACBI5TK2.0 size=1048576 2024-03-04T17:50:29.150 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.148+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~EMX8ACBI5TK2.1 size=156978 2024-03-04T17:50:29.150 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.148+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=EMX8ACBI5TK2 tag=0NH9FIBUQRB8 type=write size=2254130 2024-03-04T17:50:29.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.152+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=MZ4YZVV05MS4 type=del 2024-03-04T17:50:29.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.160+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~25UZ6ZUBNE1C.0 size=1048576 2024-03-04T17:50:29.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.164+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~25UZ6ZUBNE1C.1 size=1048576 2024-03-04T17:50:29.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.164+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~25UZ6ZUBNE1C.2 size=629764 2024-03-04T17:50:29.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.164+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=25UZ6ZUBNE1C tag=JHNHJXO9EA2F type=write size=3775492 2024-03-04T17:50:29.174 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.172+0000 7fe960f568c0 1 < suggested remove operation key=KYSLL05AUHAC tag=HBL5PVJMWKK8 type=del 2024-03-04T17:50:29.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.176+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=M7ZGI8JQMJ1L type=del size=2462444 2024-03-04T17:50:29.181 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.180+0000 7fe960f568c0 1 < completed delete operation key=ASC735ZCBLCG tag=H37DDRWHRHVY type=del 2024-03-04T17:50:29.188 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.188+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~UV6V8CW2HNB5.0 size=1048576 2024-03-04T17:50:29.191 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.188+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~UV6V8CW2HNB5.1 size=1048576 2024-03-04T17:50:29.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.192+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~UV6V8CW2HNB5.2 size=770008 2024-03-04T17:50:29.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.192+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=UV6V8CW2HNB5 tag=2MOKET05K1FD type=write size=3915736 2024-03-04T17:50:29.201 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.200+0000 7fe960f568c0 1 < suggested update operation key=JJ7SM1Y73A4C tag=3R6XHJC0CLMH type=del size=2461604 2024-03-04T17:50:29.207 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.208+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~IHQ5IF7BJXI9.0 size=1048576 2024-03-04T17:50:29.210 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.208+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~IHQ5IF7BJXI9.1 size=1048576 2024-03-04T17:50:29.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.212+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~IHQ5IF7BJXI9.2 size=187310 2024-03-04T17:50:29.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.212+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=IHQ5IF7BJXI9 tag=UIIB7Z02572K type=write size=3333038 2024-03-04T17:50:29.217 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.216+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=9YSXQJ7TUE7R type=del 2024-03-04T17:50:29.220 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.220+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=YPXLAX5Z3IJS type=del size=1825953 2024-03-04T17:50:29.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.224+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=U0443TMAGCRI type=del 2024-03-04T17:50:29.228 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.228+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=JZO8QOYC6WPZ type=write size=695926 2024-03-04T17:50:29.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.236+0000 7fe960f568c0 1 < suggested update operation key=2MKQA5JYWU8L tag=P5LPAP3IO4TL type=del size=1134058 2024-03-04T17:50:29.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.240+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=PUQVHEIPEKI0 type=write size=1033844 2024-03-04T17:50:29.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.244+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~5IJL84I8LH2V.0 size=290923 2024-03-04T17:50:29.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.248+0000 7fe960f568c0 1 < completed multipart upload key=2MKQA5JYWU8L upload=5IJL84I8LH2V tag=PU1U3L6BN3AA type=write size=1339499 2024-03-04T17:50:29.253 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.252+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=UVOK4N493U2W type=write size=585986 2024-03-04T17:50:29.257 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.256+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~PXBPPPVRY68J.0 size=924054 2024-03-04T17:50:29.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.264+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=PXBPPPVRY68J tag=MX1IWBMIUY35 type=write size=1972630 2024-03-04T17:50:29.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.268+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=S3TZJGXDU9GJ type=del size=1301963 2024-03-04T17:50:29.271 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.272+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=5LDPCAFQ41UW type=del 2024-03-04T17:50:29.275 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.276+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=GNH31WSMUP7S type=del size=2005304 2024-03-04T17:50:29.279 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.280+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~CGB11GSDIZYP.0 size=1048576 2024-03-04T17:50:29.284 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.280+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~CGB11GSDIZYP.1 size=462556 2024-03-04T17:50:29.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.288+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=CGB11GSDIZYP tag=SC6V5D22SW3O type=write size=2559708 2024-03-04T17:50:29.292 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.292+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=2J53X2DRF333 type=write size=134236 2024-03-04T17:50:29.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.296+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~32RYHL5HMSV7.0 size=1048576 2024-03-04T17:50:29.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.296+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~32RYHL5HMSV7.1 size=1048576 2024-03-04T17:50:29.301 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.300+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~32RYHL5HMSV7.2 size=482354 2024-03-04T17:50:29.307 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.308+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=32RYHL5HMSV7 tag=B0BXQFWAH0NJ type=write size=3628082 2024-03-04T17:50:29.314 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.312+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~8ES17LO9ZPAN.0 size=853836 2024-03-04T17:50:29.314 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.312+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=8ES17LO9ZPAN tag=8ARKVY077YOR type=write size=1902412 2024-03-04T17:50:29.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.320+0000 7fe960f568c0 1 < suggested remove operation key=6X4ZV3BFZNDL tag=ZF1C7FCHEXPI type=write 2024-03-04T17:50:29.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.324+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=SLMFHJQO7YGY type=del size=898534 2024-03-04T17:50:29.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.328+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=QXLXCNDKF04Q type=del 2024-03-04T17:50:29.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.332+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=O0QPAYTBKC1N type=write size=379049 2024-03-04T17:50:29.337 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.336+0000 7fe960f568c0 1 < completed write operation key=7Z7BPZ8XGBHZ tag=6720FO4K0YOZ type=write size=850251 2024-03-04T17:50:29.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.344+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~HS13CEDG8CSH.0 size=1048576 2024-03-04T17:50:29.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.348+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~HS13CEDG8CSH.1 size=1048576 2024-03-04T17:50:29.350 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.348+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~HS13CEDG8CSH.2 size=134887 2024-03-04T17:50:29.350 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.348+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=HS13CEDG8CSH tag=92PWBG0ORD83 type=write size=3280615 2024-03-04T17:50:29.351 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.352+0000 7fe960f568c0 1 < canceled part key=_multipart_RII14ZGX2EL1.2~EMX8ACBI5TK2.0 size=1048576 2024-03-04T17:50:29.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.352+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~EMX8ACBI5TK2.1 size=156978 2024-03-04T17:50:29.360 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.360+0000 7fe960f568c0 1 < canceled multipart upload key=RII14ZGX2EL1 upload=EMX8ACBI5TK2 tag=0NH9FIBUQRB8 type=write size=2254130 2024-03-04T17:50:29.365 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.364+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=YQU92X8GTXQN type=del size=612661 2024-03-04T17:50:29.369 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.368+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~25UZ6ZUBNE1C.0 size=1048576 2024-03-04T17:50:29.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.372+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~25UZ6ZUBNE1C.1 size=1048576 2024-03-04T17:50:29.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.376+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~25UZ6ZUBNE1C.2 size=629764 2024-03-04T17:50:29.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.380+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=25UZ6ZUBNE1C tag=JHNHJXO9EA2F type=write size=3775492 2024-03-04T17:50:29.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.384+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=IGG5Y1K2473I type=del size=1762824 2024-03-04T17:50:29.392 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.392+0000 7fe960f568c0 1 < canceled operation key=3X4719GHK3BI tag=M7ZGI8JQMJ1L type=del size=2462444 2024-03-04T17:50:29.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.400+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=YQM9HVJA79UO type=del size=2135127 2024-03-04T17:50:29.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.404+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~UV6V8CW2HNB5.0 size=1048576 2024-03-04T17:50:29.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.404+0000 7fe960f568c0 1 < canceled part key=_multipart_QFF67JIW8Y8E.2~UV6V8CW2HNB5.1 size=1048576 2024-03-04T17:50:29.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.408+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~UV6V8CW2HNB5.2 size=770008 2024-03-04T17:50:29.414 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.412+0000 7fe960f568c0 1 < canceled multipart upload key=QFF67JIW8Y8E upload=UV6V8CW2HNB5 tag=2MOKET05K1FD type=write size=3915736 2024-03-04T17:50:29.417 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.416+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=WNRG8F99Y35L type=del size=1284580 2024-03-04T17:50:29.421 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.420+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~IHQ5IF7BJXI9.0 size=1048576 2024-03-04T17:50:29.425 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.424+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~IHQ5IF7BJXI9.1 size=1048576 2024-03-04T17:50:29.427 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.428+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~IHQ5IF7BJXI9.2 size=187310 2024-03-04T17:50:29.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.432+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=IHQ5IF7BJXI9 tag=UIIB7Z02572K type=write size=3333038 2024-03-04T17:50:29.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.436+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=KL6ZZK71EAGT type=write size=562092 2024-03-04T17:50:29.441 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.440+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=YPXLAX5Z3IJS type=del 2024-03-04T17:50:29.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.444+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=QQSUERQUKME0 type=del size=746423 2024-03-04T17:50:29.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.452+0000 7fe960f568c0 1 < suggested update operation key=JP3FRB6MD7B4 tag=JZO8QOYC6WPZ type=write size=695926 2024-03-04T17:50:29.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.464+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~UNBKCZO9CW4J.0 size=98576 2024-03-04T17:50:29.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.464+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=UNBKCZO9CW4J tag=N64J6X453JPF type=write size=1147152 2024-03-04T17:50:29.469 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.468+0000 7fe960f568c0 1 < completed write operation key=FLYDZHOIWO6L tag=PUQVHEIPEKI0 type=write size=1033844 2024-03-04T17:50:29.472 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.472+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=8TY6XEAVTN3G type=del size=721852 2024-03-04T17:50:29.476 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.476+0000 7fe960f568c0 1 < completed write operation key=RII14ZGX2EL1 tag=UVOK4N493U2W type=write size=585986 2024-03-04T17:50:29.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.480+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~LI34Y5PT231E.0 size=1048576 2024-03-04T17:50:29.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.484+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~LI34Y5PT231E.1 size=1048576 2024-03-04T17:50:29.488 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.488+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~LI34Y5PT231E.2 size=101242 2024-03-04T17:50:29.488 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.488+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=LI34Y5PT231E tag=9VO4RCBL0OG3 type=write size=3246970 2024-03-04T17:50:29.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.492+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=S3TZJGXDU9GJ type=del 2024-03-04T17:50:29.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.496+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=D3AJ1F5QQRNT type=del size=1699506 2024-03-04T17:50:29.500 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.500+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=GNH31WSMUP7S type=del 2024-03-04T17:50:29.504 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.504+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=52ZIITKDQK4J type=del size=567897 2024-03-04T17:50:29.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.512+0000 7fe960f568c0 1 < suggested update operation key=JP3FRB6MD7B4 tag=2J53X2DRF333 type=write size=134236 2024-03-04T17:50:29.515 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.516+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=3Q9F5SCO0KP6 type=del size=935771 2024-03-04T17:50:29.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.520+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~8ES17LO9ZPAN.0 size=853836 2024-03-04T17:50:29.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.524+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=8ES17LO9ZPAN tag=8ARKVY077YOR type=write size=1902412 2024-03-04T17:50:29.532 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.532+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~IZFCWTMD7LA3.0 size=12211 2024-03-04T17:50:29.532 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.532+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=IZFCWTMD7LA3 tag=II3R22RIIH51 type=write size=1060787 2024-03-04T17:50:29.536 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.536+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=SLMFHJQO7YGY type=del 2024-03-04T17:50:29.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.540+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~3XVSI7IJEBCS.0 size=1048576 2024-03-04T17:50:29.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.544+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~3XVSI7IJEBCS.1 size=329078 2024-03-04T17:50:29.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.544+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=3XVSI7IJEBCS tag=JMFTE723AUIC type=write size=2426230 2024-03-04T17:50:29.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.552+0000 7fe960f568c0 1 < suggested update operation key=FSWBX523X9V5 tag=O0QPAYTBKC1N type=write size=379049 2024-03-04T17:50:29.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.556+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~16XEFJ4LP6B9.0 size=1048576 2024-03-04T17:50:29.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.560+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~16XEFJ4LP6B9.1 size=108167 2024-03-04T17:50:29.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.560+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=16XEFJ4LP6B9 tag=HNLTI7QV00ZK type=write size=2205319 2024-03-04T17:50:29.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.568+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~HS13CEDG8CSH.0 size=1048576 2024-03-04T17:50:29.570 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.568+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~HS13CEDG8CSH.1 size=1048576 2024-03-04T17:50:29.574 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.572+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~HS13CEDG8CSH.2 size=134887 2024-03-04T17:50:29.580 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.580+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=HS13CEDG8CSH tag=92PWBG0ORD83 type=write size=3280615 2024-03-04T17:50:29.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.584+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=B6G4F4JJN0V1 type=del size=2712890 2024-03-04T17:50:29.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.588+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=YQU92X8GTXQN type=del 2024-03-04T17:50:29.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.592+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~Z4NLYRMCE52O.0 size=1048576 2024-03-04T17:50:29.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.596+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~Z4NLYRMCE52O.1 size=448858 2024-03-04T17:50:29.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.596+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=Z4NLYRMCE52O tag=SEEVDTGRT1HE type=write size=2546010 2024-03-04T17:50:29.605 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.604+0000 7fe960f568c0 1 < suggested update operation key=8PZD01OOX3AF tag=IGG5Y1K2473I type=del size=1762824 2024-03-04T17:50:29.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.612+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~QS3Q1Y4YU0BN.0 size=1048576 2024-03-04T17:50:29.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.612+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~QS3Q1Y4YU0BN.1 size=1048576 2024-03-04T17:50:29.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.616+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~QS3Q1Y4YU0BN.2 size=16709 2024-03-04T17:50:29.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.616+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=QS3Q1Y4YU0BN tag=1MXLQAV9YBTL type=write size=3162437 2024-03-04T17:50:29.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.620+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=YQM9HVJA79UO type=del 2024-03-04T17:50:29.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.628+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~QWGQBNEF67S8.0 size=668323 2024-03-04T17:50:29.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.628+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=QWGQBNEF67S8 tag=8BTXPOEZS3SJ type=write size=1716899 2024-03-04T17:50:29.633 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.632+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=WNRG8F99Y35L type=del 2024-03-04T17:50:29.637 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.636+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=ZRQEHSA2LGKR type=del size=2570992 2024-03-04T17:50:29.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.644+0000 7fe960f568c0 1 < suggested update operation key=7Z7BPZ8XGBHZ tag=KL6ZZK71EAGT type=write size=562092 2024-03-04T17:50:29.648 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.648+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=FEBTNEWWGYX8 type=del size=2015959 2024-03-04T17:50:29.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.652+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=QQSUERQUKME0 type=del 2024-03-04T17:50:29.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.656+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=1YG58VPZ102P type=write size=333557 2024-03-04T17:50:29.660 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.660+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~UNBKCZO9CW4J.0 size=98576 2024-03-04T17:50:29.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.664+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=UNBKCZO9CW4J tag=N64J6X453JPF type=write size=1147152 2024-03-04T17:50:29.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.668+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=P4W93XNAQVAQ type=del size=2623745 2024-03-04T17:50:29.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.672+0000 7fe960f568c0 1 < canceled operation key=NF4OIJWJ83DE tag=8TY6XEAVTN3G type=del size=721852 2024-03-04T17:50:29.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.680+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~IVCG23LUG5G4.0 size=383350 2024-03-04T17:50:29.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.680+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=IVCG23LUG5G4 tag=JPPVH7V0P4DI type=write size=1431926 2024-03-04T17:50:29.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.684+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~LI34Y5PT231E.0 size=1048576 2024-03-04T17:50:29.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.684+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~LI34Y5PT231E.1 size=1048576 2024-03-04T17:50:29.692 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.692+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~LI34Y5PT231E.2 size=101242 2024-03-04T17:50:29.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.696+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=LI34Y5PT231E tag=9VO4RCBL0OG3 type=write size=3246970 2024-03-04T17:50:29.702 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.700+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=18GIKL9UURLS type=del size=1097521 2024-03-04T17:50:29.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.708+0000 7fe960f568c0 1 < suggested remove operation key=MIZRUP4MEA0J tag=D3AJ1F5QQRNT type=del 2024-03-04T17:50:29.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.712+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=1SD8WQ5HF7CW type=del size=3856009 2024-03-04T17:50:29.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.720+0000 7fe960f568c0 1 < suggested remove operation key=WNXVI6K5MU3R tag=52ZIITKDQK4J type=del 2024-03-04T17:50:29.728 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.728+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~CL60OIVNUYUL.0 size=1048576 2024-03-04T17:50:29.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.728+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~CL60OIVNUYUL.1 size=989676 2024-03-04T17:50:29.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.728+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=CL60OIVNUYUL tag=YUGV1MM5R090 type=write size=3086828 2024-03-04T17:50:29.735 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.732+0000 7fe960f568c0 1 < canceled operation key=UEVMIPQ9FR81 tag=3Q9F5SCO0KP6 type=del size=935771 2024-03-04T17:50:29.739 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.736+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=6FL78STZ38SZ type=del size=3986618 2024-03-04T17:50:29.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.744+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~IZFCWTMD7LA3.0 size=12211 2024-03-04T17:50:29.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.748+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=IZFCWTMD7LA3 tag=II3R22RIIH51 type=write size=1060787 2024-03-04T17:50:29.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.752+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=1LV3VRKQJ3XH type=del size=1246474 2024-03-04T17:50:29.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.756+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~3XVSI7IJEBCS.0 size=1048576 2024-03-04T17:50:29.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.764+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~3XVSI7IJEBCS.1 size=329078 2024-03-04T17:50:29.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.772+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=3XVSI7IJEBCS tag=JMFTE723AUIC type=write size=2426230 2024-03-04T17:50:29.776 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.776+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=XR8Q5O9XVFUO type=del size=3943227 2024-03-04T17:50:29.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.776+0000 7fe960f568c0 1 < canceled part key=_multipart_7Z7BPZ8XGBHZ.2~16XEFJ4LP6B9.0 size=1048576 2024-03-04T17:50:29.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.780+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~16XEFJ4LP6B9.1 size=108167 2024-03-04T17:50:29.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.784+0000 7fe960f568c0 1 < canceled multipart upload key=7Z7BPZ8XGBHZ upload=16XEFJ4LP6B9 tag=HNLTI7QV00ZK type=write size=2205319 2024-03-04T17:50:29.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.792+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~GC1CHEH5U0A4.0 size=727954 2024-03-04T17:50:29.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.792+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=GC1CHEH5U0A4 tag=W8AWR12GGURY type=write size=1776530 2024-03-04T17:50:29.798 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.796+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=B6G4F4JJN0V1 type=del 2024-03-04T17:50:29.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.804+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~FXG21PQKM0J4.0 size=314580 2024-03-04T17:50:29.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.804+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=FXG21PQKM0J4 tag=X4Z97UCDA7EB type=write size=1363156 2024-03-04T17:50:29.809 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.808+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~Z4NLYRMCE52O.0 size=1048576 2024-03-04T17:50:29.809 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.808+0000 7fe960f568c0 1 < canceled part key=_multipart_UEVMIPQ9FR81.2~Z4NLYRMCE52O.1 size=448858 2024-03-04T17:50:29.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.812+0000 7fe960f568c0 1 < canceled multipart upload key=UEVMIPQ9FR81 upload=Z4NLYRMCE52O tag=SEEVDTGRT1HE type=write size=2546010 2024-03-04T17:50:29.821 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.820+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~S9263X9YZZ39.0 size=1048576 2024-03-04T17:50:29.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.824+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~S9263X9YZZ39.1 size=62339 2024-03-04T17:50:29.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.824+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=S9263X9YZZ39 tag=4GUTXCWUNCD8 type=write size=2159491 2024-03-04T17:50:29.829 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.828+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~QS3Q1Y4YU0BN.0 size=1048576 2024-03-04T17:50:29.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.832+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~QS3Q1Y4YU0BN.1 size=1048576 2024-03-04T17:50:29.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.832+0000 7fe960f568c0 1 < canceled part key=_multipart_R6ZDVOUE9NXJ.2~QS3Q1Y4YU0BN.2 size=16709 2024-03-04T17:50:29.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.836+0000 7fe960f568c0 1 < canceled multipart upload key=R6ZDVOUE9NXJ upload=QS3Q1Y4YU0BN tag=1MXLQAV9YBTL type=write size=3162437 2024-03-04T17:50:29.844 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.844+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~3VAJLMHSJ7G9.0 size=1048576 2024-03-04T17:50:29.847 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.844+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~3VAJLMHSJ7G9.1 size=632102 2024-03-04T17:50:29.847 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.844+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=3VAJLMHSJ7G9 tag=E2LT61OR7D90 type=write size=2729254 2024-03-04T17:50:29.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.848+0000 7fe960f568c0 1 < canceled part key=_multipart_QGLN27WH1N2O.2~QWGQBNEF67S8.0 size=668323 2024-03-04T17:50:29.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.852+0000 7fe960f568c0 1 < canceled multipart upload key=QGLN27WH1N2O upload=QWGQBNEF67S8 tag=8BTXPOEZS3SJ type=write size=1716899 2024-03-04T17:50:29.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.856+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=RWIJYHO90YJ9 type=del size=385261 2024-03-04T17:50:29.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.860+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=ZRQEHSA2LGKR type=del 2024-03-04T17:50:29.867 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.864+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=FK07AVEYLV7O type=write size=660780 2024-03-04T17:50:29.871 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.868+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=FEBTNEWWGYX8 type=del 2024-03-04T17:50:29.875 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.872+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=RZLE0GUV43K4 type=del size=306534 2024-03-04T17:50:29.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.876+0000 7fe960f568c0 1 < completed write operation key=NF4OIJWJ83DE tag=1YG58VPZ102P type=write size=333557 2024-03-04T17:50:29.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.880+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=8INHN6GC6VC5 type=del size=2391411 2024-03-04T17:50:29.887 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.884+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=P4W93XNAQVAQ type=del 2024-03-04T17:50:29.891 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.888+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=GY0BRR4ZP9L6 type=del size=572168 2024-03-04T17:50:29.895 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.892+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~IVCG23LUG5G4.0 size=383350 2024-03-04T17:50:29.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.900+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=IVCG23LUG5G4 tag=JPPVH7V0P4DI type=write size=1431926 2024-03-04T17:50:29.904 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.904+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=GJ37JD9QTUJC type=del size=787601 2024-03-04T17:50:29.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.908+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=18GIKL9UURLS type=del 2024-03-04T17:50:29.916 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.916+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~WWNCE11A86Y1.0 size=1048576 2024-03-04T17:50:29.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.916+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~WWNCE11A86Y1.1 size=757332 2024-03-04T17:50:29.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.916+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=WWNCE11A86Y1 tag=H8X6E0OFL109 type=write size=2854484 2024-03-04T17:50:29.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.920+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=1SD8WQ5HF7CW type=del 2024-03-04T17:50:29.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.928+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~1PM2XH19NL9R.0 size=1048576 2024-03-04T17:50:29.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.932+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~1PM2XH19NL9R.1 size=1048576 2024-03-04T17:50:29.935 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.936+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~1PM2XH19NL9R.2 size=317919 2024-03-04T17:50:29.935 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.936+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=1PM2XH19NL9R tag=2YT8CHK8VJ8I type=write size=3463647 2024-03-04T17:50:29.936 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.936+0000 7fe960f568c0 1 < canceled part key=_multipart_ASC735ZCBLCG.2~CL60OIVNUYUL.0 size=1048576 2024-03-04T17:50:29.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.940+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~CL60OIVNUYUL.1 size=989676 2024-03-04T17:50:29.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.944+0000 7fe960f568c0 1 < canceled multipart upload key=ASC735ZCBLCG upload=CL60OIVNUYUL tag=YUGV1MM5R090 type=write size=3086828 2024-03-04T17:50:29.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.948+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=02CMIEHY5LF0 type=del size=720439 2024-03-04T17:50:29.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.956+0000 7fe960f568c0 1 < suggested remove operation key=RII14ZGX2EL1 tag=6FL78STZ38SZ type=del 2024-03-04T17:50:29.962 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.960+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~33AVMYD7LVZD.0 size=1044666 2024-03-04T17:50:29.962 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.960+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=33AVMYD7LVZD tag=44TTEIKK74YX type=write size=2093242 2024-03-04T17:50:29.966 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.964+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=1LV3VRKQJ3XH type=del 2024-03-04T17:50:29.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.968+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=O3JKXNNR2LF4 type=write size=776877 2024-03-04T17:50:29.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.972+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=XR8Q5O9XVFUO type=del 2024-03-04T17:50:29.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.976+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=GMDIUO7DGKUD type=del size=3385677 2024-03-04T17:50:29.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.980+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~GC1CHEH5U0A4.0 size=727954 2024-03-04T17:50:29.987 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.984+0000 7fe960f568c0 1 < completed multipart upload key=ASC735ZCBLCG upload=GC1CHEH5U0A4 tag=W8AWR12GGURY type=write size=1776530 2024-03-04T17:50:29.990 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.988+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=PO5T6VVBSXCH type=del size=788830 2024-03-04T17:50:29.994 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:29.992+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~FXG21PQKM0J4.0 size=314580 2024-03-04T17:50:30.000 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.000+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=FXG21PQKM0J4 tag=X4Z97UCDA7EB type=write size=1363156 2024-03-04T17:50:30.004 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.004+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=5BAFSBCDPYDN type=write size=186226 2024-03-04T17:50:30.007 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.008+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~S9263X9YZZ39.0 size=1048576 2024-03-04T17:50:30.010 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.008+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~S9263X9YZZ39.1 size=62339 2024-03-04T17:50:30.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.016+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=S9263X9YZZ39 tag=4GUTXCWUNCD8 type=write size=2159491 2024-03-04T17:50:30.023 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.024+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~CXP6JANDBPEV.0 size=1048576 2024-03-04T17:50:30.026 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.024+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~CXP6JANDBPEV.1 size=47178 2024-03-04T17:50:30.026 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.024+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=CXP6JANDBPEV tag=KTCFCUH5RUIZ type=write size=2144330 2024-03-04T17:50:30.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.028+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~3VAJLMHSJ7G9.0 size=1048576 2024-03-04T17:50:30.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.032+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~3VAJLMHSJ7G9.1 size=632102 2024-03-04T17:50:30.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.040+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=3VAJLMHSJ7G9 tag=E2LT61OR7D90 type=write size=2729254 2024-03-04T17:50:30.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.044+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~MK8QOE2SDD36.0 size=1048576 2024-03-04T17:50:30.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.048+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~MK8QOE2SDD36.1 size=1048576 2024-03-04T17:50:30.052 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.052+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~MK8QOE2SDD36.2 size=426573 2024-03-04T17:50:30.052 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.052+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=MK8QOE2SDD36 tag=193EARS63LT6 type=write size=3572301 2024-03-04T17:50:30.055 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.056+0000 7fe960f568c0 1 < completed delete operation key=ASC735ZCBLCG tag=RWIJYHO90YJ9 type=del 2024-03-04T17:50:30.059 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.056+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=3CVO7ZRNMMGM type=del size=3645494 2024-03-04T17:50:30.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.060+0000 7fe960f568c0 1 < completed write operation key=JP3FRB6MD7B4 tag=FK07AVEYLV7O type=write size=660780 2024-03-04T17:50:30.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.068+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~RLJEO35P9ZT7.0 size=1016584 2024-03-04T17:50:30.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.068+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=RLJEO35P9ZT7 tag=93MA1G7CEKCT type=write size=2065160 2024-03-04T17:50:30.078 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.076+0000 7fe960f568c0 1 < suggested update operation key=2FUTMG0SSXBY tag=RZLE0GUV43K4 type=del size=306534 2024-03-04T17:50:30.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.084+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~E3GPDS2RL3RU.0 size=576948 2024-03-04T17:50:30.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.084+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=E3GPDS2RL3RU tag=LLPKVRPBIFTH type=write size=1625524 2024-03-04T17:50:30.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.088+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=8INHN6GC6VC5 type=del 2024-03-04T17:50:30.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.092+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=7Y7HUU6FT7NR type=del size=3269497 2024-03-04T17:50:30.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.096+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=GY0BRR4ZP9L6 type=del 2024-03-04T17:50:30.101 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.100+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=44WJIA2CQZQW type=write size=308636 2024-03-04T17:50:30.105 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.104+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=GJ37JD9QTUJC type=del 2024-03-04T17:50:30.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.108+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=DVZ3UQGRWL0W type=del size=756500 2024-03-04T17:50:30.112 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.112+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~WWNCE11A86Y1.0 size=1048576 2024-03-04T17:50:30.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.116+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~WWNCE11A86Y1.1 size=757332 2024-03-04T17:50:30.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.124+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=WWNCE11A86Y1 tag=H8X6E0OFL109 type=write size=2854484 2024-03-04T17:50:30.127 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.128+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=QZT297C3M8GX type=write size=94861 2024-03-04T17:50:30.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.128+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~1PM2XH19NL9R.0 size=1048576 2024-03-04T17:50:30.134 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.132+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~1PM2XH19NL9R.1 size=1048576 2024-03-04T17:50:30.136 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.136+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~1PM2XH19NL9R.2 size=317919 2024-03-04T17:50:30.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.140+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=1PM2XH19NL9R tag=2YT8CHK8VJ8I type=write size=3463647 2024-03-04T17:50:30.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.144+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=ODH2DV6LIMAU type=del size=71379 2024-03-04T17:50:30.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.148+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=02CMIEHY5LF0 type=del 2024-03-04T17:50:30.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.152+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~B32W7B5W8O92.0 size=1048576 2024-03-04T17:50:30.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.156+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~B32W7B5W8O92.1 size=36676 2024-03-04T17:50:30.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.156+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=B32W7B5W8O92 tag=DR5ERWS9RKZG type=write size=2133828 2024-03-04T17:50:30.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.160+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~33AVMYD7LVZD.0 size=1044666 2024-03-04T17:50:30.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.168+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=33AVMYD7LVZD tag=44TTEIKK74YX type=write size=2093242 2024-03-04T17:50:30.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.168+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=JLYR72I5NMNK type=del size=3868349 2024-03-04T17:50:30.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.172+0000 7fe960f568c0 1 < completed write operation key=UKZ079SVZZKG tag=O3JKXNNR2LF4 type=write size=776877 2024-03-04T17:50:30.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.176+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=PDP66KSTH7U0 type=del size=3128658 2024-03-04T17:50:30.182 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.180+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=GMDIUO7DGKUD type=del 2024-03-04T17:50:30.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.184+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=YX0RWNANL31S type=del size=2782131 2024-03-04T17:50:30.189 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.188+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=PO5T6VVBSXCH type=del 2024-03-04T17:50:30.196 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.196+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~V1XKF8Q49RSM.0 size=1048576 2024-03-04T17:50:30.199 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.196+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~V1XKF8Q49RSM.1 size=440991 2024-03-04T17:50:30.199 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.196+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=V1XKF8Q49RSM tag=5PS75W1U1KGH type=write size=2538143 2024-03-04T17:50:30.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.200+0000 7fe960f568c0 1 < completed write operation key=P28N6U79ZUV2 tag=5BAFSBCDPYDN type=write size=186226 2024-03-04T17:50:30.208 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.208+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~38V6S7PZZA4R.0 size=362454 2024-03-04T17:50:30.208 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.208+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=38V6S7PZZA4R tag=GXUL07SCXQGR type=write size=1411030 2024-03-04T17:50:30.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.208+0000 7fe960f568c0 1 < canceled part key=_multipart_2FUTMG0SSXBY.2~CXP6JANDBPEV.0 size=1048576 2024-03-04T17:50:30.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.212+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~CXP6JANDBPEV.1 size=47178 2024-03-04T17:50:30.218 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.216+0000 7fe960f568c0 1 < canceled multipart upload key=2FUTMG0SSXBY upload=CXP6JANDBPEV tag=KTCFCUH5RUIZ type=write size=2144330 2024-03-04T17:50:30.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.224+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~XSAPNIUK9MW1.0 size=1048576 2024-03-04T17:50:30.227 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.224+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~XSAPNIUK9MW1.1 size=1048576 2024-03-04T17:50:30.230 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.228+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~XSAPNIUK9MW1.2 size=1045936 2024-03-04T17:50:30.230 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.228+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=XSAPNIUK9MW1 tag=GWM0K5GK4B1Z type=write size=4191664 2024-03-04T17:50:30.233 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.232+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~MK8QOE2SDD36.0 size=1048576 2024-03-04T17:50:30.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.236+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~MK8QOE2SDD36.1 size=1048576 2024-03-04T17:50:30.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.240+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~MK8QOE2SDD36.2 size=426573 2024-03-04T17:50:30.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.244+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=MK8QOE2SDD36 tag=193EARS63LT6 type=write size=3572301 2024-03-04T17:50:30.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.248+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=E26JAB9BS2QG type=del size=1179701 2024-03-04T17:50:30.253 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.252+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=3CVO7ZRNMMGM type=del 2024-03-04T17:50:30.259 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.260+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~BELSZQ4XSI8E.0 size=1048576 2024-03-04T17:50:30.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.260+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~BELSZQ4XSI8E.1 size=118115 2024-03-04T17:50:30.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.260+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=BELSZQ4XSI8E tag=YNESQE5T344R type=write size=2215267 2024-03-04T17:50:30.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.264+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~RLJEO35P9ZT7.0 size=1016584 2024-03-04T17:50:30.272 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.272+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=RLJEO35P9ZT7 tag=93MA1G7CEKCT type=write size=2065160 2024-03-04T17:50:30.279 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.276+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~X4VX0NG5BVMX.0 size=984074 2024-03-04T17:50:30.279 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.276+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=X4VX0NG5BVMX tag=1UW1X0D5VYS2 type=write size=2032650 2024-03-04T17:50:30.283 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.280+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~E3GPDS2RL3RU.0 size=576948 2024-03-04T17:50:30.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.288+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=E3GPDS2RL3RU tag=LLPKVRPBIFTH type=write size=1625524 2024-03-04T17:50:30.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.292+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=Y4V2P2Z8TOGJ type=write size=896301 2024-03-04T17:50:30.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.296+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=7Y7HUU6FT7NR type=del 2024-03-04T17:50:30.303 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.300+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=12BYSDKW549E type=del size=1530976 2024-03-04T17:50:30.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.304+0000 7fe960f568c0 1 < completed write operation key=JT02WL9JYTNM tag=44WJIA2CQZQW type=write size=308636 2024-03-04T17:50:30.308 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.308+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=JQ1U35CYBN4U type=del size=837120 2024-03-04T17:50:30.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.312+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=DVZ3UQGRWL0W type=del 2024-03-04T17:50:30.318 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.316+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~B6K2LQRIYFSG.0 size=1048576 2024-03-04T17:50:30.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.320+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~B6K2LQRIYFSG.1 size=1048576 2024-03-04T17:50:30.324 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.324+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~B6K2LQRIYFSG.2 size=683357 2024-03-04T17:50:30.324 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.324+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=B6K2LQRIYFSG tag=T45KKN7KAW1X type=write size=3829085 2024-03-04T17:50:30.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.328+0000 7fe960f568c0 1 < completed write operation key=KYSLL05AUHAC tag=QZT297C3M8GX type=write size=94861 2024-03-04T17:50:30.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.332+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=2CBWET4ZAQYS type=del size=1273999 2024-03-04T17:50:30.335 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.336+0000 7fe960f568c0 1 < canceled operation key=FSWBX523X9V5 tag=ODH2DV6LIMAU type=del size=71379 2024-03-04T17:50:30.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.340+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~4PG7DNS0HUIX.0 size=608216 2024-03-04T17:50:30.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.340+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=4PG7DNS0HUIX tag=9URP57REFZKJ type=write size=1656792 2024-03-04T17:50:30.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.344+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~B32W7B5W8O92.0 size=1048576 2024-03-04T17:50:30.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.348+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~B32W7B5W8O92.1 size=36676 2024-03-04T17:50:30.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.352+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=B32W7B5W8O92 tag=DR5ERWS9RKZG type=write size=2133828 2024-03-04T17:50:30.359 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.356+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=K4GL7AEN7MVW type=del size=552204 2024-03-04T17:50:30.363 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.360+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=JLYR72I5NMNK type=del 2024-03-04T17:50:30.370 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.368+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~LSBVUXM9RG15.0 size=1048576 2024-03-04T17:50:30.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.372+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~LSBVUXM9RG15.1 size=1048576 2024-03-04T17:50:30.375 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.376+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~LSBVUXM9RG15.2 size=618063 2024-03-04T17:50:30.375 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.376+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=LSBVUXM9RG15 tag=595EV86RGX0M type=write size=3763791 2024-03-04T17:50:30.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.376+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=PDP66KSTH7U0 type=del 2024-03-04T17:50:30.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.384+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~C3T8SDN2BQQ5.0 size=1048576 2024-03-04T17:50:30.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.388+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~C3T8SDN2BQQ5.1 size=315474 2024-03-04T17:50:30.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.388+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=C3T8SDN2BQQ5 tag=5P30Q9PMIDX6 type=write size=2412626 2024-03-04T17:50:30.393 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.392+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=YX0RWNANL31S type=del 2024-03-04T17:50:30.397 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.396+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=JMAXZYISQNWV type=del size=1350573 2024-03-04T17:50:30.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.400+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~V1XKF8Q49RSM.0 size=1048576 2024-03-04T17:50:30.403 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.404+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~V1XKF8Q49RSM.1 size=440991 2024-03-04T17:50:30.410 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.408+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=V1XKF8Q49RSM tag=5PS75W1U1KGH type=write size=2538143 2024-03-04T17:50:30.416 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.416+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=RBJKFAW4W56E type=del size=1561247 2024-03-04T17:50:30.421 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.420+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~38V6S7PZZA4R.0 size=362454 2024-03-04T17:50:30.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.424+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=38V6S7PZZA4R tag=GXUL07SCXQGR type=write size=1411030 2024-03-04T17:50:30.430 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.428+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=JIJB04KV6O7Y type=del size=3494016 2024-03-04T17:50:30.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.432+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~XSAPNIUK9MW1.0 size=1048576 2024-03-04T17:50:30.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.436+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~XSAPNIUK9MW1.1 size=1048576 2024-03-04T17:50:30.442 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.440+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~XSAPNIUK9MW1.2 size=1045936 2024-03-04T17:50:30.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.448+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=XSAPNIUK9MW1 tag=GWM0K5GK4B1Z type=write size=4191664 2024-03-04T17:50:30.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.460+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~F7Z4RLP0Q6R4.0 size=1048576 2024-03-04T17:50:30.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.464+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~F7Z4RLP0Q6R4.1 size=514403 2024-03-04T17:50:30.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.464+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=F7Z4RLP0Q6R4 tag=4WSSPOKOQX0C type=write size=2611555 2024-03-04T17:50:30.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.468+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=E26JAB9BS2QG type=del 2024-03-04T17:50:30.474 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.472+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=D1SP4RS44C12 type=del size=2521859 2024-03-04T17:50:30.478 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.476+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~BELSZQ4XSI8E.0 size=1048576 2024-03-04T17:50:30.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.480+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~BELSZQ4XSI8E.1 size=118115 2024-03-04T17:50:30.487 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.488+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=BELSZQ4XSI8E tag=YNESQE5T344R type=write size=2215267 2024-03-04T17:50:30.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.488+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=XPNYFYJXRRL1 type=del size=1452515 2024-03-04T17:50:30.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.492+0000 7fe960f568c0 1 < canceled part key=_multipart_NF4OIJWJ83DE.2~X4VX0NG5BVMX.0 size=984074 2024-03-04T17:50:30.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.496+0000 7fe960f568c0 1 < canceled multipart upload key=NF4OIJWJ83DE upload=X4VX0NG5BVMX tag=1UW1X0D5VYS2 type=write size=2032650 2024-03-04T17:50:30.504 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.504+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~2P1THNC3NHE7.0 size=1048576 2024-03-04T17:50:30.507 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.504+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~2P1THNC3NHE7.1 size=772913 2024-03-04T17:50:30.507 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.504+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=2P1THNC3NHE7 tag=PGQ2UY92PCPO type=write size=2870065 2024-03-04T17:50:30.515 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.512+0000 7fe960f568c0 1 < suggested update operation key=KYSLL05AUHAC tag=Y4V2P2Z8TOGJ type=write size=896301 2024-03-04T17:50:30.518 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.516+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=LC2DZZ4ZWUXP type=del size=1770333 2024-03-04T17:50:30.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.520+0000 7fe960f568c0 1 < canceled operation key=R6ZDVOUE9NXJ tag=12BYSDKW549E type=del size=1530976 2024-03-04T17:50:30.526 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.524+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=9CT7TD1RRIRT type=del size=3242098 2024-03-04T17:50:30.530 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.528+0000 7fe960f568c0 -1 < failed to complete operation key=KYSLL05AUHAC tag=JQ1U35CYBN4U type=del size=837120: (22) Invalid argument 2024-03-04T17:50:30.537 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.536+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~S8LM2XLM6LSV.0 size=1048576 2024-03-04T17:50:30.540 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.540+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~S8LM2XLM6LSV.1 size=1048576 2024-03-04T17:50:30.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.540+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~S8LM2XLM6LSV.2 size=954277 2024-03-04T17:50:30.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.540+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=S8LM2XLM6LSV tag=1JKXVFAH1G0K type=write size=4100005 2024-03-04T17:50:30.544 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.544+0000 7fe960f568c0 1 < canceled part key=_multipart_768X2XKWJW1S.2~B6K2LQRIYFSG.0 size=1048576 2024-03-04T17:50:30.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.544+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~B6K2LQRIYFSG.1 size=1048576 2024-03-04T17:50:30.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.548+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~B6K2LQRIYFSG.2 size=683357 2024-03-04T17:50:30.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.556+0000 7fe960f568c0 1 < canceled multipart upload key=768X2XKWJW1S upload=B6K2LQRIYFSG tag=T45KKN7KAW1X type=write size=3829085 2024-03-04T17:50:30.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.560+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=BNEH6I8AU1K6 type=del size=3291459 2024-03-04T17:50:30.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.564+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=2CBWET4ZAQYS type=del 2024-03-04T17:50:30.570 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.568+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~6H51AWUH3PYH.0 size=811980 2024-03-04T17:50:30.570 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.568+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=6H51AWUH3PYH tag=A3TNX46SUD6R type=write size=1860556 2024-03-04T17:50:30.574 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.572+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~4PG7DNS0HUIX.0 size=608216 2024-03-04T17:50:30.580 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.580+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=4PG7DNS0HUIX tag=9URP57REFZKJ type=write size=1656792 2024-03-04T17:50:30.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.584+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~XV4UCTR32EAL.0 size=70938 2024-03-04T17:50:30.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.584+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=XV4UCTR32EAL tag=9B6ZT3RSIKWH type=write size=1119514 2024-03-04T17:50:30.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.592+0000 7fe960f568c0 1 < suggested remove operation key=RII14ZGX2EL1 tag=K4GL7AEN7MVW type=del 2024-03-04T17:50:30.601 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.600+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~36HPWV07G6QU.0 size=200911 2024-03-04T17:50:30.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.600+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=36HPWV07G6QU tag=BMOK8YYQO4OP type=write size=1249487 2024-03-04T17:50:30.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.600+0000 7fe960f568c0 1 < canceled part key=_multipart_2KZ4IFW54C5B.2~LSBVUXM9RG15.0 size=1048576 2024-03-04T17:50:30.605 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.604+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~LSBVUXM9RG15.1 size=1048576 2024-03-04T17:50:30.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.608+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~LSBVUXM9RG15.2 size=618063 2024-03-04T17:50:30.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.612+0000 7fe960f568c0 1 < canceled multipart upload key=2KZ4IFW54C5B upload=LSBVUXM9RG15 tag=595EV86RGX0M type=write size=3763791 2024-03-04T17:50:30.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.616+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=STVQH9CX9JDR type=del size=891976 2024-03-04T17:50:30.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.620+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~C3T8SDN2BQQ5.0 size=1048576 2024-03-04T17:50:30.625 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.624+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~C3T8SDN2BQQ5.1 size=315474 2024-03-04T17:50:30.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.632+0000 7fe960f568c0 1 < completed multipart upload key=2MKQA5JYWU8L upload=C3T8SDN2BQQ5 tag=5P30Q9PMIDX6 type=write size=2412626 2024-03-04T17:50:30.638 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.636+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~QNEM0ZL1V8YA.0 size=1048576 2024-03-04T17:50:30.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.640+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~QNEM0ZL1V8YA.1 size=1048576 2024-03-04T17:50:30.644 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.644+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~QNEM0ZL1V8YA.2 size=859787 2024-03-04T17:50:30.644 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.644+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=QNEM0ZL1V8YA tag=PP2Z9R5ML0DX type=write size=4005515 2024-03-04T17:50:30.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.652+0000 7fe960f568c0 1 < suggested update operation key=R6ZDVOUE9NXJ tag=JMAXZYISQNWV type=del size=1350573 2024-03-04T17:50:30.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.656+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=Y14N9EIH1UGD type=write size=3646 2024-03-04T17:50:30.661 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.660+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=RBJKFAW4W56E type=del 2024-03-04T17:50:30.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.664+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=XMRZJ9UCRZTJ type=del size=2398747 2024-03-04T17:50:30.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.672+0000 7fe960f568c0 1 < suggested remove operation key=78N8HHC4O4CG tag=JIJB04KV6O7Y type=del 2024-03-04T17:50:30.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.676+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=JUSY9F9U12CA type=del size=1790712 2024-03-04T17:50:30.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.680+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~F7Z4RLP0Q6R4.0 size=1048576 2024-03-04T17:50:30.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.680+0000 7fe960f568c0 1 < canceled part key=_multipart_KYSLL05AUHAC.2~F7Z4RLP0Q6R4.1 size=514403 2024-03-04T17:50:30.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.688+0000 7fe960f568c0 1 < canceled multipart upload key=KYSLL05AUHAC upload=F7Z4RLP0Q6R4 tag=4WSSPOKOQX0C type=write size=2611555 2024-03-04T17:50:30.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.696+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~D9ZCNC7KYBMP.0 size=1048576 2024-03-04T17:50:30.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.700+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~D9ZCNC7KYBMP.1 size=1048576 2024-03-04T17:50:30.702 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.700+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~D9ZCNC7KYBMP.2 size=619393 2024-03-04T17:50:30.702 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.700+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=D9ZCNC7KYBMP tag=XVTRPQFLM8I0 type=write size=3765121 2024-03-04T17:50:30.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.704+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=D1SP4RS44C12 type=del 2024-03-04T17:50:30.712 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.712+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~0H0Q209G8MVD.0 size=1048576 2024-03-04T17:50:30.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.716+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~0H0Q209G8MVD.1 size=1048576 2024-03-04T17:50:30.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.716+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~0H0Q209G8MVD.2 size=175299 2024-03-04T17:50:30.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.716+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=0H0Q209G8MVD tag=3RHYZ5ZD0ATS type=write size=3321027 2024-03-04T17:50:30.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.720+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=XPNYFYJXRRL1 type=del 2024-03-04T17:50:30.726 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.724+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=WWL5BVZKRECZ type=del size=2402172 2024-03-04T17:50:30.729 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.728+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~2P1THNC3NHE7.0 size=1048576 2024-03-04T17:50:30.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.732+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~2P1THNC3NHE7.1 size=772913 2024-03-04T17:50:30.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.736+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=2P1THNC3NHE7 tag=PGQ2UY92PCPO type=write size=2870065 2024-03-04T17:50:30.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.744+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~EPLKBH9IETPS.0 size=1048576 2024-03-04T17:50:30.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.748+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~EPLKBH9IETPS.1 size=972256 2024-03-04T17:50:30.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.748+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=EPLKBH9IETPS tag=H0IEV33I8L9G type=write size=3069408 2024-03-04T17:50:30.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.752+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=LC2DZZ4ZWUXP type=del 2024-03-04T17:50:30.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.756+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~NW216N7SVBUM.0 size=1048576 2024-03-04T17:50:30.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.760+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~NW216N7SVBUM.1 size=694678 2024-03-04T17:50:30.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.760+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=NW216N7SVBUM tag=JI1WGXKKT6LL type=write size=2791830 2024-03-04T17:50:30.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.768+0000 7fe960f568c0 1 < suggested remove operation key=7Z7BPZ8XGBHZ tag=9CT7TD1RRIRT type=del 2024-03-04T17:50:30.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.772+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=0RSPFRF5DM5E type=del size=3806931 2024-03-04T17:50:30.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.776+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~S8LM2XLM6LSV.0 size=1048576 2024-03-04T17:50:30.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.776+0000 7fe960f568c0 1 < canceled part key=_multipart_RII14ZGX2EL1.2~S8LM2XLM6LSV.1 size=1048576 2024-03-04T17:50:30.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.780+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~S8LM2XLM6LSV.2 size=954277 2024-03-04T17:50:30.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.784+0000 7fe960f568c0 1 < canceled multipart upload key=RII14ZGX2EL1 upload=S8LM2XLM6LSV tag=1JKXVFAH1G0K type=write size=4100005 2024-03-04T17:50:30.792 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.792+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~0AO17HC8WGHW.0 size=1048576 2024-03-04T17:50:30.795 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.792+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~0AO17HC8WGHW.1 size=1048576 2024-03-04T17:50:30.798 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.796+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~0AO17HC8WGHW.2 size=466296 2024-03-04T17:50:30.798 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.796+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=0AO17HC8WGHW tag=NV6C1PMM4RP1 type=write size=3612024 2024-03-04T17:50:30.801 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.800+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=BNEH6I8AU1K6 type=del 2024-03-04T17:50:30.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.812+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~E6HMN4CMQ65T.0 size=1048576 2024-03-04T17:50:30.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.816+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~E6HMN4CMQ65T.1 size=1048576 2024-03-04T17:50:30.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.816+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~E6HMN4CMQ65T.2 size=165812 2024-03-04T17:50:30.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.816+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=E6HMN4CMQ65T tag=DYJ0OCW3EPKT type=write size=3311540 2024-03-04T17:50:30.823 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.820+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~6H51AWUH3PYH.0 size=811980 2024-03-04T17:50:30.829 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.828+0000 7fe960f568c0 1 < completed multipart upload key=2MKQA5JYWU8L upload=6H51AWUH3PYH tag=A3TNX46SUD6R type=write size=1860556 2024-03-04T17:50:30.835 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.832+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~G2YH4J5VD8J6.0 size=1048576 2024-03-04T17:50:30.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.836+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~G2YH4J5VD8J6.1 size=855370 2024-03-04T17:50:30.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.836+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=G2YH4J5VD8J6 tag=SZVASWRQDLDR type=write size=2952522 2024-03-04T17:50:30.842 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.840+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~XV4UCTR32EAL.0 size=70938 2024-03-04T17:50:30.847 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.848+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=XV4UCTR32EAL tag=9B6ZT3RSIKWH type=write size=1119514 2024-03-04T17:50:30.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.852+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~NMNVJPKWJJYL.0 size=1048576 2024-03-04T17:50:30.857 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.856+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~NMNVJPKWJJYL.1 size=570391 2024-03-04T17:50:30.857 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.856+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=NMNVJPKWJJYL tag=GHBI1FQ64X6G type=write size=2667543 2024-03-04T17:50:30.861 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.860+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~36HPWV07G6QU.0 size=200911 2024-03-04T17:50:30.866 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.864+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=36HPWV07G6QU tag=BMOK8YYQO4OP type=write size=1249487 2024-03-04T17:50:30.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.868+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=UEKE0QS3TNW1 type=write size=516827 2024-03-04T17:50:30.874 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.872+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=STVQH9CX9JDR type=del 2024-03-04T17:50:30.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.876+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=G9JHE30D7A2O type=del size=1739013 2024-03-04T17:50:30.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.880+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~QNEM0ZL1V8YA.0 size=1048576 2024-03-04T17:50:30.886 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.884+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~QNEM0ZL1V8YA.1 size=1048576 2024-03-04T17:50:30.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.888+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~QNEM0ZL1V8YA.2 size=859787 2024-03-04T17:50:30.895 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.896+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=QNEM0ZL1V8YA tag=PP2Z9R5ML0DX type=write size=4005515 2024-03-04T17:50:30.899 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.900+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=KBN2YJRAX7LN type=del size=2724300 2024-03-04T17:50:30.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.904+0000 7fe960f568c0 1 < suggested remove operation key=WNXVI6K5MU3R tag=Y14N9EIH1UGD type=write 2024-03-04T17:50:30.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.908+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=UADRLRCQ041R type=del size=551008 2024-03-04T17:50:30.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.916+0000 7fe960f568c0 -1 < failed to complete operation key=WNXVI6K5MU3R tag=XMRZJ9UCRZTJ type=del size=2398747: (22) Invalid argument 2024-03-04T17:50:30.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.916+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=Y9RBJV88X7PY type=del size=2355195 2024-03-04T17:50:30.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.920+0000 7fe960f568c0 1 < canceled operation key=768X2XKWJW1S tag=JUSY9F9U12CA type=del size=1790712 2024-03-04T17:50:30.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.928+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~TSTH27LUK9OD.0 size=168343 2024-03-04T17:50:30.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.928+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=TSTH27LUK9OD tag=W86KPVZ6MJ4B type=write size=1216919 2024-03-04T17:50:30.933 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.932+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~D9ZCNC7KYBMP.0 size=1048576 2024-03-04T17:50:30.936 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.936+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~D9ZCNC7KYBMP.1 size=1048576 2024-03-04T17:50:30.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.940+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~D9ZCNC7KYBMP.2 size=619393 2024-03-04T17:50:30.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.944+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=D9ZCNC7KYBMP tag=XVTRPQFLM8I0 type=write size=3765121 2024-03-04T17:50:30.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.952+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=M9CSXHDQZVOS type=write size=682389 2024-03-04T17:50:30.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.952+0000 7fe960f568c0 1 < canceled part key=_multipart_JP3FRB6MD7B4.2~0H0Q209G8MVD.0 size=1048576 2024-03-04T17:50:30.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.956+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~0H0Q209G8MVD.1 size=1048576 2024-03-04T17:50:30.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.956+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~0H0Q209G8MVD.2 size=175299 2024-03-04T17:50:30.965 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.964+0000 7fe960f568c0 1 < canceled multipart upload key=JP3FRB6MD7B4 upload=0H0Q209G8MVD tag=3RHYZ5ZD0ATS type=write size=3321027 2024-03-04T17:50:30.972 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.972+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~B9PLDLBYHT2M.0 size=378365 2024-03-04T17:50:30.972 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.972+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=B9PLDLBYHT2M tag=PGGM9T8JC1PN type=write size=1426941 2024-03-04T17:50:30.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.976+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=WWL5BVZKRECZ type=del 2024-03-04T17:50:31.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.980+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~9WUVTSREGIA2.0 size=1048576 2024-03-04T17:50:31.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.984+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~9WUVTSREGIA2.1 size=1048576 2024-03-04T17:50:31.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.988+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~9WUVTSREGIA2.2 size=705006 2024-03-04T17:50:31.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.988+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=9WUVTSREGIA2 tag=CODBCU53WXFP type=write size=3850734 2024-03-04T17:50:31.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.992+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~EPLKBH9IETPS.0 size=1048576 2024-03-04T17:50:31.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:30.992+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~EPLKBH9IETPS.1 size=972256 2024-03-04T17:50:31.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.000+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=EPLKBH9IETPS tag=H0IEV33I8L9G type=write size=3069408 2024-03-04T17:50:31.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.004+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=9JME97AJDPWY type=write size=1041169 2024-03-04T17:50:31.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.008+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~NW216N7SVBUM.0 size=1048576 2024-03-04T17:50:31.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.012+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~NW216N7SVBUM.1 size=694678 2024-03-04T17:50:31.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.016+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=NW216N7SVBUM tag=JI1WGXKKT6LL type=write size=2791830 2024-03-04T17:50:31.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.024+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~Q1OZ3GAUAGUJ.0 size=1048576 2024-03-04T17:50:31.026 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.024+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~Q1OZ3GAUAGUJ.1 size=475787 2024-03-04T17:50:31.027 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.024+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=Q1OZ3GAUAGUJ tag=LGQKKT2XNA4M type=write size=2572939 2024-03-04T17:50:31.031 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.028+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=0RSPFRF5DM5E type=del 2024-03-04T17:50:31.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.032+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=V3SL71DB8CDL type=del size=1896468 2024-03-04T17:50:31.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.036+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~0AO17HC8WGHW.0 size=1048576 2024-03-04T17:50:31.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.040+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~0AO17HC8WGHW.1 size=1048576 2024-03-04T17:50:31.044 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.044+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~0AO17HC8WGHW.2 size=466296 2024-03-04T17:50:31.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.048+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=0AO17HC8WGHW tag=NV6C1PMM4RP1 type=write size=3612024 2024-03-04T17:50:31.054 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.052+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=67HN856BIWTK type=del size=2789730 2024-03-04T17:50:31.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.056+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~E6HMN4CMQ65T.0 size=1048576 2024-03-04T17:50:31.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.060+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~E6HMN4CMQ65T.1 size=1048576 2024-03-04T17:50:31.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.064+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~E6HMN4CMQ65T.2 size=165812 2024-03-04T17:50:31.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.072+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=E6HMN4CMQ65T tag=DYJ0OCW3EPKT type=write size=3311540 2024-03-04T17:50:31.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.076+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=K20P16L6YY0E type=del size=2885040 2024-03-04T17:50:31.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.076+0000 7fe960f568c0 1 < canceled part key=_multipart_R6ZDVOUE9NXJ.2~G2YH4J5VD8J6.0 size=1048576 2024-03-04T17:50:31.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.080+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~G2YH4J5VD8J6.1 size=855370 2024-03-04T17:50:31.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.084+0000 7fe960f568c0 1 < canceled multipart upload key=R6ZDVOUE9NXJ upload=G2YH4J5VD8J6 tag=SZVASWRQDLDR type=write size=2952522 2024-03-04T17:50:31.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.092+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~KCQJOKLTJF8U.0 size=890081 2024-03-04T17:50:31.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.092+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=KCQJOKLTJF8U tag=BQQNIN2MM0M4 type=write size=1938657 2024-03-04T17:50:31.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.096+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~NMNVJPKWJJYL.0 size=1048576 2024-03-04T17:50:31.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.096+0000 7fe960f568c0 1 < canceled part key=_multipart_R6ZDVOUE9NXJ.2~NMNVJPKWJJYL.1 size=570391 2024-03-04T17:50:31.103 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.104+0000 7fe960f568c0 1 < canceled multipart upload key=R6ZDVOUE9NXJ upload=NMNVJPKWJJYL tag=GHBI1FQ64X6G type=write size=2667543 2024-03-04T17:50:31.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.108+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=A2EQZJSAUFUZ type=del size=2705072 2024-03-04T17:50:31.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.112+0000 7fe960f568c0 1 < completed write operation key=NF4OIJWJ83DE tag=UEKE0QS3TNW1 type=write size=516827 2024-03-04T17:50:31.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.116+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=WLA041TI93UV type=del size=3371193 2024-03-04T17:50:31.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.120+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=G9JHE30D7A2O type=del 2024-03-04T17:50:31.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.124+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=F54Z3R69B3K3 type=write size=773203 2024-03-04T17:50:31.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.128+0000 7fe960f568c0 1 < suggested remove operation key=GZ4RRIR5CC26 tag=KBN2YJRAX7LN type=del 2024-03-04T17:50:31.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.136+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~LSAN6VSO0YJE.0 size=511205 2024-03-04T17:50:31.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.136+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=LSAN6VSO0YJE tag=HR59PQ2SHFDG type=write size=1559781 2024-03-04T17:50:31.141 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.140+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=UADRLRCQ041R type=del 2024-03-04T17:50:31.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.144+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=9UVZ2IH0263R type=del size=3549378 2024-03-04T17:50:31.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.148+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=Y9RBJV88X7PY type=del 2024-03-04T17:50:31.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.156+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=0OBRTM0J3P5Q type=write size=529805 2024-03-04T17:50:31.159 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.156+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~TSTH27LUK9OD.0 size=168343 2024-03-04T17:50:31.169 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.168+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=TSTH27LUK9OD tag=W86KPVZ6MJ4B type=write size=1216919 2024-03-04T17:50:31.174 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.172+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=JLQKPCJ8MG3L type=write size=530642 2024-03-04T17:50:31.182 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.180+0000 7fe960f568c0 1 < suggested update operation key=JT02WL9JYTNM tag=M9CSXHDQZVOS type=write size=682389 2024-03-04T17:50:31.189 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.188+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~NM0KAGCPSLYK.0 size=803685 2024-03-04T17:50:31.190 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.188+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=NM0KAGCPSLYK tag=GQMND6Z6ABF3 type=write size=1852261 2024-03-04T17:50:31.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.192+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~B9PLDLBYHT2M.0 size=378365 2024-03-04T17:50:31.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.200+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=B9PLDLBYHT2M tag=PGGM9T8JC1PN type=write size=1426941 2024-03-04T17:50:31.204 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.204+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=7EKB8CVJVFC6 type=write size=620090 2024-03-04T17:50:31.208 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.208+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~9WUVTSREGIA2.0 size=1048576 2024-03-04T17:50:31.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.208+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~9WUVTSREGIA2.1 size=1048576 2024-03-04T17:50:31.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.212+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~9WUVTSREGIA2.2 size=705006 2024-03-04T17:50:31.220 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.220+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=9WUVTSREGIA2 tag=CODBCU53WXFP type=write size=3850734 2024-03-04T17:50:31.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.224+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=AE8IX5AT9FGA type=del size=3493555 2024-03-04T17:50:31.228 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.228+0000 7fe960f568c0 1 < completed write operation key=BFF9OD00TU54 tag=9JME97AJDPWY type=write size=1041169 2024-03-04T17:50:31.232 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.232+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=5CS5GJ83PO9Z type=del size=4160063 2024-03-04T17:50:31.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.236+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~Q1OZ3GAUAGUJ.0 size=1048576 2024-03-04T17:50:31.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.236+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~Q1OZ3GAUAGUJ.1 size=475787 2024-03-04T17:50:31.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.244+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=Q1OZ3GAUAGUJ tag=LGQKKT2XNA4M type=write size=2572939 2024-03-04T17:50:31.247 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.248+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=HVYUWYQ0YSWP type=del size=1320545 2024-03-04T17:50:31.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.252+0000 7fe960f568c0 1 < suggested remove operation key=2MKQA5JYWU8L tag=V3SL71DB8CDL type=del 2024-03-04T17:50:31.259 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.256+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=9LH2E3QCX3RX type=del size=2339740 2024-03-04T17:50:31.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.264+0000 7fe960f568c0 1 < suggested remove operation key=JSO6WIYMPCF6 tag=67HN856BIWTK type=del 2024-03-04T17:50:31.270 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.268+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=TX6A9QP910A0 type=write size=915199 2024-03-04T17:50:31.277 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.276+0000 7fe960f568c0 1 < suggested update operation key=BFF9OD00TU54 tag=K20P16L6YY0E type=del size=2885040 2024-03-04T17:50:31.281 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.280+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=OAMELEIWSW6X type=write size=835269 2024-03-04T17:50:31.285 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.284+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~KCQJOKLTJF8U.0 size=890081 2024-03-04T17:50:31.291 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.288+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=KCQJOKLTJF8U tag=BQQNIN2MM0M4 type=write size=1938657 2024-03-04T17:50:31.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.292+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=41TJAJGXYV17 type=del size=2652237 2024-03-04T17:50:31.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.296+0000 7fe960f568c0 -1 < failed to complete operation key=JT02WL9JYTNM tag=A2EQZJSAUFUZ type=del size=2705072: (22) Invalid argument 2024-03-04T17:50:31.302 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.300+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=ZR4FCHJ38HQQ type=del size=1472714 2024-03-04T17:50:31.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.304+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=WLA041TI93UV type=del 2024-03-04T17:50:31.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.308+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=2T9YXPI2MSKG type=write size=1005009 2024-03-04T17:50:31.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.312+0000 7fe960f568c0 1 < completed write operation key=XSQ3GE2UHMZG tag=F54Z3R69B3K3 type=write size=773203 2024-03-04T17:50:31.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.316+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=6JTMRCUOR137 type=del size=2135887 2024-03-04T17:50:31.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.320+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~LSAN6VSO0YJE.0 size=511205 2024-03-04T17:50:31.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.328+0000 7fe960f568c0 1 < completed multipart upload key=ASC735ZCBLCG upload=LSAN6VSO0YJE tag=HR59PQ2SHFDG type=write size=1559781 2024-03-04T17:50:31.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.332+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=X16CG6V0RJY9 type=write size=860700 2024-03-04T17:50:31.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.340+0000 7fe960f568c0 1 < suggested update operation key=768X2XKWJW1S tag=9UVZ2IH0263R type=del size=3549378 2024-03-04T17:50:31.348 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.352+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~1HH7R9YVMNOE.0 size=179398 2024-03-04T17:50:31.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.352+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=1HH7R9YVMNOE tag=I5Y5734MLQ5I type=write size=1227974 2024-03-04T17:50:31.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.352+0000 7fe960f568c0 1 < canceled operation key=R6ZDVOUE9NXJ tag=0OBRTM0J3P5Q type=write size=529805 2024-03-04T17:50:31.361 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.360+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~GWZ71W9DNUJH.0 size=547156 2024-03-04T17:50:31.361 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.360+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=GWZ71W9DNUJH tag=1C9FKAJUUZU7 type=write size=1595732 2024-03-04T17:50:31.365 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.364+0000 7fe960f568c0 1 < completed write operation key=3YTAYUQLLUKY tag=JLQKPCJ8MG3L type=write size=530642 2024-03-04T17:50:31.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.372+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~WJZY9WYXGW6A.0 size=1048576 2024-03-04T17:50:31.375 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.376+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~WJZY9WYXGW6A.1 size=1048576 2024-03-04T17:50:31.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.376+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~WJZY9WYXGW6A.2 size=826609 2024-03-04T17:50:31.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.376+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=WJZY9WYXGW6A tag=8NLV2A3ASLES type=write size=3972337 2024-03-04T17:50:31.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.380+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~NM0KAGCPSLYK.0 size=803685 2024-03-04T17:50:31.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.388+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=NM0KAGCPSLYK tag=GQMND6Z6ABF3 type=write size=1852261 2024-03-04T17:50:31.393 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.392+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=OMU42U0V0QOJ type=write size=644246 2024-03-04T17:50:31.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.396+0000 7fe960f568c0 1 < completed write operation key=3X4719GHK3BI tag=7EKB8CVJVFC6 type=write size=620090 2024-03-04T17:50:31.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.400+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=VG15UBIGR558 type=write size=45727 2024-03-04T17:50:31.406 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.404+0000 7fe960f568c0 -1 < failed to complete operation key=2MKQA5JYWU8L tag=AE8IX5AT9FGA type=del size=3493555: (22) Invalid argument 2024-03-04T17:50:31.410 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.408+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=0ZYOPSQ9LFF2 type=del size=2678905 2024-03-04T17:50:31.414 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.412+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=5CS5GJ83PO9Z type=del 2024-03-04T17:50:31.421 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.420+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~C378Y5ZP523F.0 size=623063 2024-03-04T17:50:31.422 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.420+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=C378Y5ZP523F tag=SJC1PP637HL2 type=write size=1671639 2024-03-04T17:50:31.425 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.424+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=HVYUWYQ0YSWP type=del 2024-03-04T17:50:31.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.432+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~S9VVWV26M1V7.0 size=371201 2024-03-04T17:50:31.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.432+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=S9VVWV26M1V7 tag=5I3I7CTPXEVA type=write size=1419777 2024-03-04T17:50:31.436 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.436+0000 7fe960f568c0 -1 < failed to complete operation key=768X2XKWJW1S tag=9LH2E3QCX3RX type=del size=2339740: (22) Invalid argument 2024-03-04T17:50:31.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.440+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=QMP072W6KEHV type=del size=1805875 2024-03-04T17:50:31.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.444+0000 7fe960f568c0 1 < suggested update operation key=P28N6U79ZUV2 tag=TX6A9QP910A0 type=write size=915199 2024-03-04T17:50:31.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.448+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=AQ735LU6O6I0 type=del size=3480539 2024-03-04T17:50:31.454 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.452+0000 7fe960f568c0 1 < completed write operation key=JJ7SM1Y73A4C tag=OAMELEIWSW6X type=write size=835269 2024-03-04T17:50:31.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.460+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~70LETB3Q4D7N.0 size=693190 2024-03-04T17:50:31.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.460+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=70LETB3Q4D7N tag=IWCKS0C1P4JC type=write size=1741766 2024-03-04T17:50:31.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.464+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=41TJAJGXYV17 type=del 2024-03-04T17:50:31.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.472+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~1G3SDY0QZJGQ.0 size=38869 2024-03-04T17:50:31.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.472+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=1G3SDY0QZJGQ tag=TR2PFMME9TIY type=write size=1087445 2024-03-04T17:50:31.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.476+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=ZR4FCHJ38HQQ type=del 2024-03-04T17:50:31.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.480+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~J8Z377J1I5DE.0 size=1048576 2024-03-04T17:50:31.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.484+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~J8Z377J1I5DE.1 size=643086 2024-03-04T17:50:31.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.484+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=J8Z377J1I5DE tag=9MZIDBKMLR00 type=write size=2740238 2024-03-04T17:50:31.489 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.488+0000 7fe960f568c0 1 < completed write operation key=2FUTMG0SSXBY tag=2T9YXPI2MSKG type=write size=1005009 2024-03-04T17:50:31.494 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.492+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=YXZXCZP430OD type=del size=456772 2024-03-04T17:50:31.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.496+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=6JTMRCUOR137 type=del 2024-03-04T17:50:31.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.500+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=GIZVYKH5GVJ5 type=del size=3978580 2024-03-04T17:50:31.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.504+0000 7fe960f568c0 1 < completed write operation key=2FUTMG0SSXBY tag=X16CG6V0RJY9 type=write size=860700 2024-03-04T17:50:31.512 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.512+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~JKDTG82WX4CX.0 size=1048576 2024-03-04T17:50:31.515 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.512+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~JKDTG82WX4CX.1 size=838946 2024-03-04T17:50:31.515 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.512+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=JKDTG82WX4CX tag=1ZXQJDD18FLP type=write size=2936098 2024-03-04T17:50:31.518 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.516+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~1HH7R9YVMNOE.0 size=179398 2024-03-04T17:50:31.527 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.524+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=1HH7R9YVMNOE tag=I5Y5734MLQ5I type=write size=1227974 2024-03-04T17:50:31.528 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.528+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=LODC54JVQLM4 type=del size=2984221 2024-03-04T17:50:31.537 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.536+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~GWZ71W9DNUJH.0 size=547156 2024-03-04T17:50:31.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.540+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=GWZ71W9DNUJH tag=1C9FKAJUUZU7 type=write size=1595732 2024-03-04T17:50:31.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.548+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~2VXSNCQKHOKW.0 size=584009 2024-03-04T17:50:31.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.548+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=2VXSNCQKHOKW tag=BIBNGKVRBVHZ type=write size=1632585 2024-03-04T17:50:31.554 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.552+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~WJZY9WYXGW6A.0 size=1048576 2024-03-04T17:50:31.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.556+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~WJZY9WYXGW6A.1 size=1048576 2024-03-04T17:50:31.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.560+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~WJZY9WYXGW6A.2 size=826609 2024-03-04T17:50:31.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.564+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=WJZY9WYXGW6A tag=8NLV2A3ASLES type=write size=3972337 2024-03-04T17:50:31.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.572+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~3QSAW7UIO2OG.0 size=1048576 2024-03-04T17:50:31.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.576+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~3QSAW7UIO2OG.1 size=684599 2024-03-04T17:50:31.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.576+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=3QSAW7UIO2OG tag=E6QKOQULJY6T type=write size=2781751 2024-03-04T17:50:31.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.580+0000 7fe960f568c0 1 < completed write operation key=768X2XKWJW1S tag=OMU42U0V0QOJ type=write size=644246 2024-03-04T17:50:31.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.580+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=D96IPJ6AO5OS type=del size=4121674 2024-03-04T17:50:31.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.584+0000 7fe960f568c0 1 < completed write operation key=3YTAYUQLLUKY tag=VG15UBIGR558 type=write size=45727 2024-03-04T17:50:31.590 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.588+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=B9WFUBTOAZCM type=del size=3614332 2024-03-04T17:50:31.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.592+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=0ZYOPSQ9LFF2 type=del 2024-03-04T17:50:31.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.596+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=712QSCX5XRP1 type=del size=2347960 2024-03-04T17:50:31.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.600+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~C378Y5ZP523F.0 size=623063 2024-03-04T17:50:31.607 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.608+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=C378Y5ZP523F tag=SJC1PP637HL2 type=write size=1671639 2024-03-04T17:50:31.611 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.612+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=CXFCR7KPJMU2 type=del size=3664996 2024-03-04T17:50:31.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.612+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~S9VVWV26M1V7.0 size=371201 2024-03-04T17:50:31.621 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.620+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=S9VVWV26M1V7 tag=5I3I7CTPXEVA type=write size=1419777 2024-03-04T17:50:31.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.628+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~8CRUI4GKF68Z.0 size=1048576 2024-03-04T17:50:31.630 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.628+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~8CRUI4GKF68Z.1 size=770707 2024-03-04T17:50:31.630 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.628+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=8CRUI4GKF68Z tag=35KFDAY275YZ type=write size=2867859 2024-03-04T17:50:31.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.632+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=QMP072W6KEHV type=del 2024-03-04T17:50:31.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.640+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~9Q1SKPSTLMSQ.0 size=1048576 2024-03-04T17:50:31.644 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.644+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~9Q1SKPSTLMSQ.1 size=839705 2024-03-04T17:50:31.644 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.644+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=9Q1SKPSTLMSQ tag=PUHNM20Q14VU type=write size=2936857 2024-03-04T17:50:31.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.652+0000 7fe960f568c0 1 < suggested remove operation key=U2G6YEVD5506 tag=AQ735LU6O6I0 type=del 2024-03-04T17:50:31.655 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.656+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=BEICRGMT8VE2 type=del size=3274941 2024-03-04T17:50:31.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.660+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~70LETB3Q4D7N.0 size=693190 2024-03-04T17:50:31.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.664+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=70LETB3Q4D7N tag=IWCKS0C1P4JC type=write size=1741766 2024-03-04T17:50:31.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.668+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=3W7MD583MI2H type=del size=678931 2024-03-04T17:50:31.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.672+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~1G3SDY0QZJGQ.0 size=38869 2024-03-04T17:50:31.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.676+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=1G3SDY0QZJGQ tag=TR2PFMME9TIY type=write size=1087445 2024-03-04T17:50:31.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.680+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=74CSKI76EJ26 type=del size=1371890 2024-03-04T17:50:31.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.684+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~J8Z377J1I5DE.0 size=1048576 2024-03-04T17:50:31.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.688+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~J8Z377J1I5DE.1 size=643086 2024-03-04T17:50:31.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.696+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=J8Z377J1I5DE tag=9MZIDBKMLR00 type=write size=2740238 2024-03-04T17:50:31.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.700+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=Q5JFLB2860WZ type=del size=1046307 2024-03-04T17:50:31.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.704+0000 7fe960f568c0 -1 < failed to complete operation key=U2G6YEVD5506 tag=YXZXCZP430OD type=del size=456772: (22) Invalid argument 2024-03-04T17:50:31.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.708+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=TWOXGT6S7VS4 type=del size=766247 2024-03-04T17:50:31.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.712+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=GIZVYKH5GVJ5 type=del 2024-03-04T17:50:31.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.716+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=LZ5U05XW4HTZ type=del size=2357052 2024-03-04T17:50:31.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.720+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~JKDTG82WX4CX.0 size=1048576 2024-03-04T17:50:31.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.720+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~JKDTG82WX4CX.1 size=838946 2024-03-04T17:50:31.728 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.728+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=JKDTG82WX4CX tag=1ZXQJDD18FLP type=write size=2936098 2024-03-04T17:50:31.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.732+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=X7YEE73E82CB type=write size=820783 2024-03-04T17:50:31.736 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.736+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=LODC54JVQLM4 type=del 2024-03-04T17:50:31.740 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.740+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=0B6TW4ZAJK65 type=del size=3727436 2024-03-04T17:50:31.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.740+0000 7fe960f568c0 1 < canceled part key=_multipart_U2G6YEVD5506.2~2VXSNCQKHOKW.0 size=584009 2024-03-04T17:50:31.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.744+0000 7fe960f568c0 1 < canceled multipart upload key=U2G6YEVD5506 upload=2VXSNCQKHOKW tag=BIBNGKVRBVHZ type=write size=1632585 2024-03-04T17:50:31.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.748+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=F4B3XK93SCC3 type=del size=3687862 2024-03-04T17:50:31.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.752+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~3QSAW7UIO2OG.0 size=1048576 2024-03-04T17:50:31.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.756+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~3QSAW7UIO2OG.1 size=684599 2024-03-04T17:50:31.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.764+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=3QSAW7UIO2OG tag=E6QKOQULJY6T type=write size=2781751 2024-03-04T17:50:31.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.768+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~DMLY3MOKRCYO.0 size=989801 2024-03-04T17:50:31.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.768+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=DMLY3MOKRCYO tag=W133TJQRXBKH type=write size=2038377 2024-03-04T17:50:31.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.772+0000 7fe960f568c0 1 < canceled operation key=MIZRUP4MEA0J tag=D96IPJ6AO5OS type=del size=4121674 2024-03-04T17:50:31.781 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.780+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~KILS0F0XMILS.0 size=1048576 2024-03-04T17:50:31.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.784+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~KILS0F0XMILS.1 size=794016 2024-03-04T17:50:31.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.784+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=KILS0F0XMILS tag=YT7R5ZXY1XGK type=write size=2891168 2024-03-04T17:50:31.788 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.788+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=B9WFUBTOAZCM type=del 2024-03-04T17:50:31.792 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.792+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=OPCDG8ZH6LK3 type=del size=4108649 2024-03-04T17:50:31.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.796+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=712QSCX5XRP1 type=del 2024-03-04T17:50:31.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.800+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=RWP5F4LNYGMV type=del size=3809816 2024-03-04T17:50:31.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.804+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=CXFCR7KPJMU2 type=del 2024-03-04T17:50:31.809 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.808+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=2B770CVZQ18A type=del size=616003 2024-03-04T17:50:31.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.812+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~8CRUI4GKF68Z.0 size=1048576 2024-03-04T17:50:31.816 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.816+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~8CRUI4GKF68Z.1 size=770707 2024-03-04T17:50:31.823 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.820+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=8CRUI4GKF68Z tag=35KFDAY275YZ type=write size=2867859 2024-03-04T17:50:31.830 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.828+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~ZMEZXNWHDSLJ.0 size=1048576 2024-03-04T17:50:31.833 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.832+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~ZMEZXNWHDSLJ.1 size=1048576 2024-03-04T17:50:31.835 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.836+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~ZMEZXNWHDSLJ.2 size=823700 2024-03-04T17:50:31.835 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.836+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=ZMEZXNWHDSLJ tag=021E1H0XJNV1 type=write size=3969428 2024-03-04T17:50:31.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.840+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~9Q1SKPSTLMSQ.0 size=1048576 2024-03-04T17:50:31.843 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.840+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~9Q1SKPSTLMSQ.1 size=839705 2024-03-04T17:50:31.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.848+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=9Q1SKPSTLMSQ tag=PUHNM20Q14VU type=write size=2936857 2024-03-04T17:50:31.853 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.852+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=EWXTK3ZF60ZI type=write size=855222 2024-03-04T17:50:31.861 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.860+0000 7fe960f568c0 1 < suggested remove operation key=7Z7BPZ8XGBHZ tag=BEICRGMT8VE2 type=del 2024-03-04T17:50:31.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.864+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=3X0M4RB062EA type=del size=128855 2024-03-04T17:50:31.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.868+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=3W7MD583MI2H type=del 2024-03-04T17:50:31.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.872+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=B030156B6LBH type=write size=580135 2024-03-04T17:50:31.880 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.880+0000 7fe960f568c0 1 < suggested update operation key=NF4OIJWJ83DE tag=74CSKI76EJ26 type=del size=1371890 2024-03-04T17:50:31.892 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.892+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~HC038XT32AVS.0 size=1048576 2024-03-04T17:50:31.895 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.892+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~HC038XT32AVS.1 size=1048576 2024-03-04T17:50:31.898 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.896+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~HC038XT32AVS.2 size=529253 2024-03-04T17:50:31.898 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.896+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=HC038XT32AVS tag=4EOICXEG09Y3 type=write size=3674981 2024-03-04T17:50:31.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.904+0000 7fe960f568c0 1 < suggested remove operation key=JSO6WIYMPCF6 tag=Q5JFLB2860WZ type=del 2024-03-04T17:50:31.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.908+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=7NSB8A2SVBLX type=del size=3957101 2024-03-04T17:50:31.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.912+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=TWOXGT6S7VS4 type=del 2024-03-04T17:50:31.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.920+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~DDVT25G1H63X.0 size=1048576 2024-03-04T17:50:31.925 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.924+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~DDVT25G1H63X.1 size=1048576 2024-03-04T17:50:31.928 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.928+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~DDVT25G1H63X.2 size=530938 2024-03-04T17:50:31.928 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.928+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=DDVT25G1H63X tag=IEG7X4TSYFEL type=write size=3676666 2024-03-04T17:50:31.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.932+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=LZ5U05XW4HTZ type=del 2024-03-04T17:50:31.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.936+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~QA3OMXN3N8UB.0 size=308999 2024-03-04T17:50:31.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.936+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=QA3OMXN3N8UB tag=4KUD2176UCL1 type=write size=1357575 2024-03-04T17:50:31.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.940+0000 7fe960f568c0 1 < completed write operation key=BFF9OD00TU54 tag=X7YEE73E82CB type=write size=820783 2024-03-04T17:50:31.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.948+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~4216JPY3WVZD.0 size=1048576 2024-03-04T17:50:31.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.952+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~4216JPY3WVZD.1 size=1048576 2024-03-04T17:50:31.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.952+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~4216JPY3WVZD.2 size=778459 2024-03-04T17:50:31.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.952+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=4216JPY3WVZD tag=M8AWPUIH4QT1 type=write size=3924187 2024-03-04T17:50:31.959 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.956+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=0B6TW4ZAJK65 type=del 2024-03-04T17:50:31.962 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.960+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=XXAD8UKRCM6X type=del size=177682 2024-03-04T17:50:31.966 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.964+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=F4B3XK93SCC3 type=del 2024-03-04T17:50:31.973 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.972+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~KGBDSOPDY89X.0 size=1048576 2024-03-04T17:50:31.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.976+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~KGBDSOPDY89X.1 size=1048576 2024-03-04T17:50:31.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.976+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~KGBDSOPDY89X.2 size=420966 2024-03-04T17:50:31.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.976+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=KGBDSOPDY89X tag=0JNG8OANZJF3 type=write size=3566694 2024-03-04T17:50:31.983 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.980+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~DMLY3MOKRCYO.0 size=989801 2024-03-04T17:50:31.989 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.988+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=DMLY3MOKRCYO tag=W133TJQRXBKH type=write size=2038377 2024-03-04T17:50:31.992 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.992+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=NJ6FFZKYLCOS type=write size=311388 2024-03-04T17:50:31.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:31.996+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~KILS0F0XMILS.0 size=1048576 2024-03-04T17:50:31.999 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.000+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~KILS0F0XMILS.1 size=794016 2024-03-04T17:50:32.005 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.004+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=KILS0F0XMILS tag=YT7R5ZXY1XGK type=write size=2891168 2024-03-04T17:50:32.009 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.008+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=LYHR8GKSURND type=del size=2749213 2024-03-04T17:50:32.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.012+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=OPCDG8ZH6LK3 type=del 2024-03-04T17:50:32.018 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.016+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=GPBCOHS9IFE8 type=del size=2197580 2024-03-04T17:50:32.022 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.020+0000 7fe960f568c0 1 < completed delete operation key=U2G6YEVD5506 tag=RWP5F4LNYGMV type=del 2024-03-04T17:50:32.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.024+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=3SYKA6D55ZJ8 type=del size=1320604 2024-03-04T17:50:32.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.028+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=2B770CVZQ18A type=del 2024-03-04T17:50:32.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.036+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~7W154Y8K853G.0 size=1048576 2024-03-04T17:50:32.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.036+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~7W154Y8K853G.1 size=724495 2024-03-04T17:50:32.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.036+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=7W154Y8K853G tag=LTEWQKFQ630T type=write size=2821647 2024-03-04T17:50:32.043 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.040+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~ZMEZXNWHDSLJ.0 size=1048576 2024-03-04T17:50:32.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.044+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~ZMEZXNWHDSLJ.1 size=1048576 2024-03-04T17:50:32.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.048+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~ZMEZXNWHDSLJ.2 size=823700 2024-03-04T17:50:32.055 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.056+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=ZMEZXNWHDSLJ tag=021E1H0XJNV1 type=write size=3969428 2024-03-04T17:50:32.059 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.060+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=0TD2M8NTSUJB type=del size=933658 2024-03-04T17:50:32.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.064+0000 7fe960f568c0 1 < completed write operation key=2KZ4IFW54C5B tag=EWXTK3ZF60ZI type=write size=855222 2024-03-04T17:50:32.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.068+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~075NB61KM06P.0 size=1048576 2024-03-04T17:50:32.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.072+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~075NB61KM06P.1 size=292355 2024-03-04T17:50:32.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.072+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=075NB61KM06P tag=MI135E1JBWQZ type=write size=2389507 2024-03-04T17:50:32.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.080+0000 7fe960f568c0 1 < suggested remove operation key=WNXVI6K5MU3R tag=3X0M4RB062EA type=del 2024-03-04T17:50:32.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.084+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~F7JXVDPF4G02.0 size=472630 2024-03-04T17:50:32.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.084+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=F7JXVDPF4G02 tag=26KIWLYLPQAK type=write size=1521206 2024-03-04T17:50:32.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.088+0000 7fe960f568c0 1 < completed write operation key=JP3FRB6MD7B4 tag=B030156B6LBH type=write size=580135 2024-03-04T17:50:32.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.096+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~9FZOLMB08Q6M.0 size=474184 2024-03-04T17:50:32.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.096+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=9FZOLMB08Q6M tag=L3RD3KY1ZVDY type=write size=1522760 2024-03-04T17:50:32.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.100+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~HC038XT32AVS.0 size=1048576 2024-03-04T17:50:32.104 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.104+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~HC038XT32AVS.1 size=1048576 2024-03-04T17:50:32.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.104+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~HC038XT32AVS.2 size=529253 2024-03-04T17:50:32.112 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.112+0000 7fe960f568c0 1 < completed multipart upload key=U2G6YEVD5506 upload=HC038XT32AVS tag=4EOICXEG09Y3 type=write size=3674981 2024-03-04T17:50:32.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.116+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~8BQLX6IAV71D.0 size=1048576 2024-03-04T17:50:32.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.120+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~8BQLX6IAV71D.1 size=475333 2024-03-04T17:50:32.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.120+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=8BQLX6IAV71D tag=PKUYCNPTKN9Y type=write size=2572485 2024-03-04T17:50:32.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.124+0000 7fe960f568c0 1 < canceled operation key=FLYDZHOIWO6L tag=7NSB8A2SVBLX type=del size=3957101 2024-03-04T17:50:32.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.128+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=T2GQ11MX06NH type=del size=3207583 2024-03-04T17:50:32.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.128+0000 7fe960f568c0 1 < canceled part key=_multipart_8PZD01OOX3AF.2~DDVT25G1H63X.0 size=1048576 2024-03-04T17:50:32.133 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.132+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~DDVT25G1H63X.1 size=1048576 2024-03-04T17:50:32.136 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.136+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~DDVT25G1H63X.2 size=530938 2024-03-04T17:50:32.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.140+0000 7fe960f568c0 1 < canceled multipart upload key=8PZD01OOX3AF upload=DDVT25G1H63X tag=IEG7X4TSYFEL type=write size=3676666 2024-03-04T17:50:32.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.148+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~OENOFQ1O9WHM.0 size=1048576 2024-03-04T17:50:32.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.152+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~OENOFQ1O9WHM.1 size=1048576 2024-03-04T17:50:32.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.152+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~OENOFQ1O9WHM.2 size=532756 2024-03-04T17:50:32.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.152+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=OENOFQ1O9WHM tag=FACI8E2P30O6 type=write size=3678484 2024-03-04T17:50:32.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.156+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~QA3OMXN3N8UB.0 size=308999 2024-03-04T17:50:32.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.164+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=QA3OMXN3N8UB tag=4KUD2176UCL1 type=write size=1357575 2024-03-04T17:50:32.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.168+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=KSPH3498GFRL type=del size=3199715 2024-03-04T17:50:32.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.168+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~4216JPY3WVZD.0 size=1048576 2024-03-04T17:50:32.174 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.172+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~4216JPY3WVZD.1 size=1048576 2024-03-04T17:50:32.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.176+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~4216JPY3WVZD.2 size=778459 2024-03-04T17:50:32.182 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.180+0000 7fe960f568c0 1 < completed multipart upload key=U2G6YEVD5506 upload=4216JPY3WVZD tag=M8AWPUIH4QT1 type=write size=3924187 2024-03-04T17:50:32.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.184+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=KBG6B72SWC35 type=del size=3829692 2024-03-04T17:50:32.190 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.188+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=XXAD8UKRCM6X type=del 2024-03-04T17:50:32.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.192+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=5G26WUYT763O type=del size=3163693 2024-03-04T17:50:32.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.196+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~KGBDSOPDY89X.0 size=1048576 2024-03-04T17:50:32.201 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.200+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~KGBDSOPDY89X.1 size=1048576 2024-03-04T17:50:32.203 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.204+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~KGBDSOPDY89X.2 size=420966 2024-03-04T17:50:32.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.208+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=KGBDSOPDY89X tag=0JNG8OANZJF3 type=write size=3566694 2024-03-04T17:50:32.215 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.212+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~PHRAKEBV2XT5.0 size=186075 2024-03-04T17:50:32.215 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.212+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=PHRAKEBV2XT5 tag=39MOLIG9LO6M type=write size=1234651 2024-03-04T17:50:32.218 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.216+0000 7fe960f568c0 1 < completed write operation key=7Z7BPZ8XGBHZ tag=NJ6FFZKYLCOS type=write size=311388 2024-03-04T17:50:32.225 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.224+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~EAUJ89H20I62.0 size=1048576 2024-03-04T17:50:32.232 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.232+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~EAUJ89H20I62.1 size=1048576 2024-03-04T17:50:32.234 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.232+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~EAUJ89H20I62.2 size=256217 2024-03-04T17:50:32.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.232+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=EAUJ89H20I62 tag=8QYXBLKZRZZZ type=write size=3401945 2024-03-04T17:50:32.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.236+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=LYHR8GKSURND type=del 2024-03-04T17:50:32.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.240+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=ZJ1G3V12BHQU type=del size=2736746 2024-03-04T17:50:32.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.244+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=GPBCOHS9IFE8 type=del 2024-03-04T17:50:32.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.248+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=G58WTL6MWFM2 type=del size=2119585 2024-03-04T17:50:32.252 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.252+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=3SYKA6D55ZJ8 type=del 2024-03-04T17:50:32.258 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.256+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~S7ZMFXQMOV9H.0 size=956432 2024-03-04T17:50:32.258 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.256+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=S7ZMFXQMOV9H tag=SOYS8GYJICLL type=write size=2005008 2024-03-04T17:50:32.261 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.260+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~7W154Y8K853G.0 size=1048576 2024-03-04T17:50:32.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.264+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~7W154Y8K853G.1 size=724495 2024-03-04T17:50:32.268 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.268+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=7W154Y8K853G tag=LTEWQKFQ630T type=write size=2821647 2024-03-04T17:50:32.272 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.272+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=A1J4WIROKT37 type=del size=578821 2024-03-04T17:50:32.275 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.272+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=0TD2M8NTSUJB type=del 2024-03-04T17:50:32.281 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.280+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~60C57JAGO3IE.0 size=1048576 2024-03-04T17:50:32.284 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.284+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~60C57JAGO3IE.1 size=1048576 2024-03-04T17:50:32.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.284+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~60C57JAGO3IE.2 size=725339 2024-03-04T17:50:32.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.284+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=60C57JAGO3IE tag=WUFC6L08KSM1 type=write size=3871067 2024-03-04T17:50:32.291 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.288+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~075NB61KM06P.0 size=1048576 2024-03-04T17:50:32.291 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.288+0000 7fe960f568c0 1 < canceled part key=_multipart_RII14ZGX2EL1.2~075NB61KM06P.1 size=292355 2024-03-04T17:50:32.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.296+0000 7fe960f568c0 1 < canceled multipart upload key=RII14ZGX2EL1 upload=075NB61KM06P tag=MI135E1JBWQZ type=write size=2389507 2024-03-04T17:50:32.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.296+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=WL818EDJSVCU type=del size=1646606 2024-03-04T17:50:32.302 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.300+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~F7JXVDPF4G02.0 size=472630 2024-03-04T17:50:32.307 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.304+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=F7JXVDPF4G02 tag=26KIWLYLPQAK type=write size=1521206 2024-03-04T17:50:32.310 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.308+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=4B0JSOJGYA86 type=del size=3960651 2024-03-04T17:50:32.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.312+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~9FZOLMB08Q6M.0 size=474184 2024-03-04T17:50:32.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.316+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=9FZOLMB08Q6M tag=L3RD3KY1ZVDY type=write size=1522760 2024-03-04T17:50:32.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.320+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=FU0XIWPJQ626 type=del size=263022 2024-03-04T17:50:32.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.324+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~8BQLX6IAV71D.0 size=1048576 2024-03-04T17:50:32.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.328+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~8BQLX6IAV71D.1 size=475333 2024-03-04T17:50:32.334 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.332+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=8BQLX6IAV71D tag=PKUYCNPTKN9Y type=write size=2572485 2024-03-04T17:50:32.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.340+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~TRNP0AXLQQQZ.0 size=121167 2024-03-04T17:50:32.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.340+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=TRNP0AXLQQQZ tag=0EY1MVSFXY0I type=write size=1169743 2024-03-04T17:50:32.348 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.348+0000 7fe960f568c0 1 < suggested update operation key=3X4719GHK3BI tag=T2GQ11MX06NH type=del size=3207583 2024-03-04T17:50:32.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.352+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=CB21BW5HPJ23 type=del size=2912271 2024-03-04T17:50:32.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.356+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~OENOFQ1O9WHM.0 size=1048576 2024-03-04T17:50:32.358 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.356+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~OENOFQ1O9WHM.1 size=1048576 2024-03-04T17:50:32.361 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.360+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~OENOFQ1O9WHM.2 size=532756 2024-03-04T17:50:32.368 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.368+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=OENOFQ1O9WHM tag=FACI8E2P30O6 type=write size=3678484 2024-03-04T17:50:32.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.372+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=5RMFEPHPR0HX type=del size=1409374 2024-03-04T17:50:32.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.376+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=KSPH3498GFRL type=del 2024-03-04T17:50:32.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.380+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=79Q94U08OTL5 type=del size=3091815 2024-03-04T17:50:32.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.384+0000 7fe960f568c0 1 < canceled operation key=JP3FRB6MD7B4 tag=KBG6B72SWC35 type=del size=3829692 2024-03-04T17:50:32.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.384+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=UPDLAODMUMUY type=del size=3273564 2024-03-04T17:50:32.390 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.388+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=5G26WUYT763O type=del 2024-03-04T17:50:32.397 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.396+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~REI4X3KJZBPZ.0 size=1048576 2024-03-04T17:50:32.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.400+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~REI4X3KJZBPZ.1 size=287861 2024-03-04T17:50:32.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.400+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=REI4X3KJZBPZ tag=5GWNMFQIN6V7 type=write size=2385013 2024-03-04T17:50:32.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.400+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~PHRAKEBV2XT5.0 size=186075 2024-03-04T17:50:32.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.408+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=PHRAKEBV2XT5 tag=39MOLIG9LO6M type=write size=1234651 2024-03-04T17:50:32.412 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.412+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=G4JVF6VTDBFO type=del size=1561492 2024-03-04T17:50:32.416 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.416+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~EAUJ89H20I62.0 size=1048576 2024-03-04T17:50:32.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.420+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~EAUJ89H20I62.1 size=1048576 2024-03-04T17:50:32.422 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.420+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~EAUJ89H20I62.2 size=256217 2024-03-04T17:50:32.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.428+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=EAUJ89H20I62 tag=8QYXBLKZRZZZ type=write size=3401945 2024-03-04T17:50:32.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.432+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=GVIG3LCBR9IS type=del size=2048325 2024-03-04T17:50:32.437 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.436+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=ZJ1G3V12BHQU type=del 2024-03-04T17:50:32.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.444+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~IJR4GNDIXZCT.0 size=257845 2024-03-04T17:50:32.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.444+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=IJR4GNDIXZCT tag=3KSK3637W27N type=write size=1306421 2024-03-04T17:50:32.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.444+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=G58WTL6MWFM2 type=del 2024-03-04T17:50:32.451 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.448+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=5DWRCW59406L type=del size=3271838 2024-03-04T17:50:32.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.452+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~S7ZMFXQMOV9H.0 size=956432 2024-03-04T17:50:32.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.460+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=S7ZMFXQMOV9H tag=SOYS8GYJICLL type=write size=2005008 2024-03-04T17:50:32.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.464+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=OCZ88MPT8FKY type=del size=3100049 2024-03-04T17:50:32.468 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.468+0000 7fe960f568c0 -1 < failed to complete operation key=3X4719GHK3BI tag=A1J4WIROKT37 type=del size=578821: (22) Invalid argument 2024-03-04T17:50:32.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.476+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~6B39BDF7GY7Z.0 size=968339 2024-03-04T17:50:32.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.476+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=6B39BDF7GY7Z tag=HY5DGB9VD2OY type=write size=2016915 2024-03-04T17:50:32.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.480+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~60C57JAGO3IE.0 size=1048576 2024-03-04T17:50:32.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.480+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~60C57JAGO3IE.1 size=1048576 2024-03-04T17:50:32.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.484+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~60C57JAGO3IE.2 size=725339 2024-03-04T17:50:32.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.492+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=60C57JAGO3IE tag=WUFC6L08KSM1 type=write size=3871067 2024-03-04T17:50:32.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.492+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=369Q48SAZGJA type=del size=973473 2024-03-04T17:50:32.503 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.500+0000 7fe960f568c0 1 < suggested update operation key=R6ZDVOUE9NXJ tag=WL818EDJSVCU type=del size=1646606 2024-03-04T17:50:32.509 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.508+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~COV22T11DHT7.0 size=1048576 2024-03-04T17:50:32.512 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.512+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~COV22T11DHT7.1 size=372287 2024-03-04T17:50:32.512 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.512+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=COV22T11DHT7 tag=6R2HHUUL9GSX type=write size=2469439 2024-03-04T17:50:32.516 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.516+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=4B0JSOJGYA86 type=del 2024-03-04T17:50:32.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.520+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=JYDHF7R26L4D type=write size=478955 2024-03-04T17:50:32.524 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.524+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=FU0XIWPJQ626 type=del 2024-03-04T17:50:32.531 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.528+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~U9F5RHZCBRTC.0 size=1048576 2024-03-04T17:50:32.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.532+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~U9F5RHZCBRTC.1 size=1048576 2024-03-04T17:50:32.536 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.536+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~U9F5RHZCBRTC.2 size=321859 2024-03-04T17:50:32.536 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.536+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=U9F5RHZCBRTC tag=T3HVQYT135IW type=write size=3467587 2024-03-04T17:50:32.540 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.540+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~TRNP0AXLQQQZ.0 size=121167 2024-03-04T17:50:32.546 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.544+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=TRNP0AXLQQQZ tag=0EY1MVSFXY0I type=write size=1169743 2024-03-04T17:50:32.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.548+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=IA3HQG7WEZWV type=del size=1628627 2024-03-04T17:50:32.553 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.552+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=CB21BW5HPJ23 type=del 2024-03-04T17:50:32.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.556+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=9JSOXQLYQPYH type=del size=1859067 2024-03-04T17:50:32.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.564+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=5RMFEPHPR0HX type=del 2024-03-04T17:50:32.573 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.572+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~IED2PATYTJFE.0 size=1048576 2024-03-04T17:50:32.576 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.576+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~IED2PATYTJFE.1 size=231097 2024-03-04T17:50:32.576 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.576+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=IED2PATYTJFE tag=0IB7UM4RH0EC type=write size=2328249 2024-03-04T17:50:32.580 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.580+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=79Q94U08OTL5 type=del 2024-03-04T17:50:32.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.584+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~Z11FHNHLF284.0 size=1048576 2024-03-04T17:50:32.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.588+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~Z11FHNHLF284.1 size=893138 2024-03-04T17:50:32.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.588+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=Z11FHNHLF284 tag=EKAUW2B0LHUS type=write size=2990290 2024-03-04T17:50:32.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.592+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=UPDLAODMUMUY type=del 2024-03-04T17:50:32.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.596+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=NTKD0LH3U80X type=del size=908090 2024-03-04T17:50:32.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.600+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~REI4X3KJZBPZ.0 size=1048576 2024-03-04T17:50:32.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.604+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~REI4X3KJZBPZ.1 size=287861 2024-03-04T17:50:32.609 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.608+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=REI4X3KJZBPZ tag=5GWNMFQIN6V7 type=write size=2385013 2024-03-04T17:50:32.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.612+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=83H9OXJ46QE4 type=del size=688 2024-03-04T17:50:32.617 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.616+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=G4JVF6VTDBFO type=del 2024-03-04T17:50:32.624 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.624+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~Z8JS2LWQZXYT.0 size=1048576 2024-03-04T17:50:32.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.628+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~Z8JS2LWQZXYT.1 size=309724 2024-03-04T17:50:32.628 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.628+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=Z8JS2LWQZXYT tag=JVM6R9HAAGX4 type=write size=2406876 2024-03-04T17:50:32.632 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.632+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=GVIG3LCBR9IS type=del 2024-03-04T17:50:32.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.640+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~NRW3152U394J.0 size=1048576 2024-03-04T17:50:32.644 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.644+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~NRW3152U394J.1 size=1048576 2024-03-04T17:50:32.648 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.648+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~NRW3152U394J.2 size=389519 2024-03-04T17:50:32.648 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.648+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=NRW3152U394J tag=S5KJZKG3EQ6O type=write size=3535247 2024-03-04T17:50:32.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.652+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~IJR4GNDIXZCT.0 size=257845 2024-03-04T17:50:32.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.656+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=IJR4GNDIXZCT tag=3KSK3637W27N type=write size=1306421 2024-03-04T17:50:32.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.664+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~PS9CN7ASPIC4.0 size=1048576 2024-03-04T17:50:32.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.668+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~PS9CN7ASPIC4.1 size=1048576 2024-03-04T17:50:32.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.672+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~PS9CN7ASPIC4.2 size=162584 2024-03-04T17:50:32.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.672+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=PS9CN7ASPIC4 tag=D8TC9M2J3FX0 type=write size=3308312 2024-03-04T17:50:32.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.676+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=5DWRCW59406L type=del 2024-03-04T17:50:32.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.684+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~23S49HDCQHJM.0 size=1048576 2024-03-04T17:50:32.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.688+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~23S49HDCQHJM.1 size=1048576 2024-03-04T17:50:32.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.688+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~23S49HDCQHJM.2 size=246065 2024-03-04T17:50:32.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.688+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=23S49HDCQHJM tag=373MKLXV6MKW type=write size=3391793 2024-03-04T17:50:32.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.692+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=OCZ88MPT8FKY type=del 2024-03-04T17:50:32.702 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.700+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~TAPRMAOHT5NU.0 size=1048576 2024-03-04T17:50:32.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.704+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~TAPRMAOHT5NU.1 size=911483 2024-03-04T17:50:32.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.704+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=TAPRMAOHT5NU tag=5LJFDPOQ154W type=write size=3008635 2024-03-04T17:50:32.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.708+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~6B39BDF7GY7Z.0 size=968339 2024-03-04T17:50:32.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.716+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=6B39BDF7GY7Z tag=HY5DGB9VD2OY type=write size=2016915 2024-03-04T17:50:32.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.720+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=MJFZ1PEFNX6R type=del size=3147736 2024-03-04T17:50:32.725 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.724+0000 7fe960f568c0 -1 < failed to complete operation key=R6ZDVOUE9NXJ tag=369Q48SAZGJA type=del size=973473: (22) Invalid argument 2024-03-04T17:50:32.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.732+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~75VJB1O0LINF.0 size=1005384 2024-03-04T17:50:32.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.732+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=75VJB1O0LINF tag=Z3Z3ABRQBK31 type=write size=2053960 2024-03-04T17:50:32.736 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.736+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~COV22T11DHT7.0 size=1048576 2024-03-04T17:50:32.739 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.736+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~COV22T11DHT7.1 size=372287 2024-03-04T17:50:32.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.744+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=COV22T11DHT7 tag=6R2HHUUL9GSX type=write size=2469439 2024-03-04T17:50:32.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.752+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~1X7SI9M5BKQ2.0 size=1048576 2024-03-04T17:50:32.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.756+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~1X7SI9M5BKQ2.1 size=821003 2024-03-04T17:50:32.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.756+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=1X7SI9M5BKQ2 tag=T5YG66O75M7U type=write size=2918155 2024-03-04T17:50:32.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.756+0000 7fe960f568c0 1 < completed write operation key=8PZD01OOX3AF tag=JYDHF7R26L4D type=write size=478955 2024-03-04T17:50:32.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.760+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=IK9DLNU51P00 type=del size=740373 2024-03-04T17:50:32.767 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.768+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~U9F5RHZCBRTC.0 size=1048576 2024-03-04T17:50:32.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.768+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~U9F5RHZCBRTC.1 size=1048576 2024-03-04T17:50:32.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.772+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~U9F5RHZCBRTC.2 size=321859 2024-03-04T17:50:32.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.780+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=U9F5RHZCBRTC tag=T3HVQYT135IW type=write size=3467587 2024-03-04T17:50:32.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.780+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=W236DLQLJBWY type=del size=346652 2024-03-04T17:50:32.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.784+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=IA3HQG7WEZWV type=del 2024-03-04T17:50:32.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.788+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=IV7LRJN9JG77 type=del size=2892906 2024-03-04T17:50:32.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.792+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=9JSOXQLYQPYH type=del 2024-03-04T17:50:32.797 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.796+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=XW2F33ATRG2Y type=del size=3825649 2024-03-04T17:50:32.801 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.800+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~IED2PATYTJFE.0 size=1048576 2024-03-04T17:50:32.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.804+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~IED2PATYTJFE.1 size=231097 2024-03-04T17:50:32.810 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.808+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=IED2PATYTJFE tag=0IB7UM4RH0EC type=write size=2328249 2024-03-04T17:50:32.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.812+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=ZSA9JWIHRF1I type=del size=2825455 2024-03-04T17:50:32.817 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.816+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~Z11FHNHLF284.0 size=1048576 2024-03-04T17:50:32.820 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.820+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~Z11FHNHLF284.1 size=893138 2024-03-04T17:50:32.826 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.824+0000 7fe960f568c0 1 < completed multipart upload key=U2G6YEVD5506 upload=Z11FHNHLF284 tag=EKAUW2B0LHUS type=write size=2990290 2024-03-04T17:50:32.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.832+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~GEGEOJYMHIWC.0 size=78667 2024-03-04T17:50:32.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.832+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=GEGEOJYMHIWC tag=Q0WBFT4AGDQB type=write size=1127243 2024-03-04T17:50:32.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.840+0000 7fe960f568c0 1 < suggested update operation key=3X4719GHK3BI tag=NTKD0LH3U80X type=del size=908090 2024-03-04T17:50:32.842 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.840+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=3CROXR9JPHO4 type=del size=2372930 2024-03-04T17:50:32.846 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.844+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=83H9OXJ46QE4 type=del 2024-03-04T17:50:32.850 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.848+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=U8F7D35OQF4I type=del size=620602 2024-03-04T17:50:32.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.852+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~Z8JS2LWQZXYT.0 size=1048576 2024-03-04T17:50:32.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.856+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~Z8JS2LWQZXYT.1 size=309724 2024-03-04T17:50:32.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.860+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=Z8JS2LWQZXYT tag=JVM6R9HAAGX4 type=write size=2406876 2024-03-04T17:50:32.866 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.864+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=3WQFX9TUBHZ3 type=write size=845999 2024-03-04T17:50:32.871 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.868+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~NRW3152U394J.0 size=1048576 2024-03-04T17:50:32.874 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.872+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~NRW3152U394J.1 size=1048576 2024-03-04T17:50:32.877 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.876+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~NRW3152U394J.2 size=389519 2024-03-04T17:50:32.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.884+0000 7fe960f568c0 1 < completed multipart upload key=GZ4RRIR5CC26 upload=NRW3152U394J tag=S5KJZKG3EQ6O type=write size=3535247 2024-03-04T17:50:32.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.888+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~W58CXF67HEG5.0 size=1048576 2024-03-04T17:50:32.893 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.892+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~W58CXF67HEG5.1 size=187877 2024-03-04T17:50:32.893 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.892+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=W58CXF67HEG5 tag=7KU9SU3HWQ6F type=write size=2285029 2024-03-04T17:50:32.897 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.896+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~PS9CN7ASPIC4.0 size=1048576 2024-03-04T17:50:32.900 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.900+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~PS9CN7ASPIC4.1 size=1048576 2024-03-04T17:50:32.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.900+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~PS9CN7ASPIC4.2 size=162584 2024-03-04T17:50:32.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.908+0000 7fe960f568c0 1 < completed multipart upload key=XSQ3GE2UHMZG upload=PS9CN7ASPIC4 tag=D8TC9M2J3FX0 type=write size=3308312 2024-03-04T17:50:32.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.916+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=9J42XL4ZMSYZ type=del size=1502756 2024-03-04T17:50:32.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.920+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~23S49HDCQHJM.0 size=1048576 2024-03-04T17:50:32.925 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.924+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~23S49HDCQHJM.1 size=1048576 2024-03-04T17:50:32.928 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.928+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~23S49HDCQHJM.2 size=246065 2024-03-04T17:50:32.934 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.932+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=23S49HDCQHJM tag=373MKLXV6MKW type=write size=3391793 2024-03-04T17:50:32.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.940+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~SBKUH5QE31HS.0 size=1048576 2024-03-04T17:50:32.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.944+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~SBKUH5QE31HS.1 size=518845 2024-03-04T17:50:32.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.944+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=SBKUH5QE31HS tag=2BRRMA7B4SX2 type=write size=2615997 2024-03-04T17:50:32.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.944+0000 7fe960f568c0 1 < canceled part key=_multipart_BFF9OD00TU54.2~TAPRMAOHT5NU.0 size=1048576 2024-03-04T17:50:32.947 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.948+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~TAPRMAOHT5NU.1 size=911483 2024-03-04T17:50:32.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.952+0000 7fe960f568c0 1 < canceled multipart upload key=BFF9OD00TU54 upload=TAPRMAOHT5NU tag=5LJFDPOQ154W type=write size=3008635 2024-03-04T17:50:32.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.956+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=YY5TAZMFVUIL type=del size=1653470 2024-03-04T17:50:32.963 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.960+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=MJFZ1PEFNX6R type=del 2024-03-04T17:50:32.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.968+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~X1GPAAVWRLJM.0 size=1048576 2024-03-04T17:50:32.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.968+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~X1GPAAVWRLJM.1 size=1048576 2024-03-04T17:50:32.973 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.972+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~X1GPAAVWRLJM.2 size=85017 2024-03-04T17:50:32.973 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.972+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=X1GPAAVWRLJM tag=0ZSDFKSMG45Z type=write size=3230745 2024-03-04T17:50:32.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.976+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~75VJB1O0LINF.0 size=1005384 2024-03-04T17:50:32.983 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.980+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=75VJB1O0LINF tag=Z3Z3ABRQBK31 type=write size=2053960 2024-03-04T17:50:32.990 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.988+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~7XZJEXZLTUXF.0 size=1048576 2024-03-04T17:50:32.993 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.992+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~7XZJEXZLTUXF.1 size=499465 2024-03-04T17:50:32.993 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.992+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=7XZJEXZLTUXF tag=IIX2G8QZ089K type=write size=2596617 2024-03-04T17:50:32.997 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:32.996+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~1X7SI9M5BKQ2.0 size=1048576 2024-03-04T17:50:33.000 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.000+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~1X7SI9M5BKQ2.1 size=821003 2024-03-04T17:50:33.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.004+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=1X7SI9M5BKQ2 tag=T5YG66O75M7U type=write size=2918155 2024-03-04T17:50:33.010 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.008+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=YLKWRS6AC7GU type=del size=1324130 2024-03-04T17:50:33.013 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.012+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=IK9DLNU51P00 type=del 2024-03-04T17:50:33.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.020+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~JSMOQVPYJCGM.0 size=1048576 2024-03-04T17:50:33.024 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.028+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~JSMOQVPYJCGM.1 size=1048576 2024-03-04T17:50:33.027 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.028+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~JSMOQVPYJCGM.2 size=1041052 2024-03-04T17:50:33.027 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.028+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=JSMOQVPYJCGM tag=6JQT42W2ODKD type=write size=4186780 2024-03-04T17:50:33.031 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.032+0000 7fe960f568c0 1 < canceled operation key=8PZD01OOX3AF tag=W236DLQLJBWY type=del size=346652 2024-03-04T17:50:33.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.036+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~N53KE57QPVYT.0 size=1048576 2024-03-04T17:50:33.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.040+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~N53KE57QPVYT.1 size=911494 2024-03-04T17:50:33.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.040+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=N53KE57QPVYT tag=E0R01I015GAB type=write size=3008646 2024-03-04T17:50:33.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.044+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=IV7LRJN9JG77 type=del 2024-03-04T17:50:33.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.048+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=Q381AI9VELJP type=del size=3294165 2024-03-04T17:50:33.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.056+0000 7fe960f568c0 1 < suggested remove operation key=WNXVI6K5MU3R tag=XW2F33ATRG2Y type=del 2024-03-04T17:50:33.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.060+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=02J9K5632HTR type=write size=893289 2024-03-04T17:50:33.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.064+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=ZSA9JWIHRF1I type=del 2024-03-04T17:50:33.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.072+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~OA3233YRRDS7.0 size=1048576 2024-03-04T17:50:33.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.072+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~OA3233YRRDS7.1 size=267616 2024-03-04T17:50:33.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.072+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=OA3233YRRDS7 tag=UG8NQAM8AK00 type=write size=2364768 2024-03-04T17:50:33.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.076+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~GEGEOJYMHIWC.0 size=78667 2024-03-04T17:50:33.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.084+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=GEGEOJYMHIWC tag=Q0WBFT4AGDQB type=write size=1127243 2024-03-04T17:50:33.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.088+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=MIRKMD5XOD7L type=write size=773749 2024-03-04T17:50:33.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.092+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=3CROXR9JPHO4 type=del 2024-03-04T17:50:33.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.096+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=L0LA9F9RG528 type=write size=624704 2024-03-04T17:50:33.102 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.100+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=U8F7D35OQF4I type=del 2024-03-04T17:50:33.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.108+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~I0S4VAKUWAX4.0 size=773988 2024-03-04T17:50:33.109 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.108+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=I0S4VAKUWAX4 tag=GL9DRLEL91ST type=write size=1822564 2024-03-04T17:50:33.113 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.112+0000 7fe960f568c0 1 < completed write operation key=NF4OIJWJ83DE tag=3WQFX9TUBHZ3 type=write size=845999 2024-03-04T17:50:33.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.116+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=VQTB5CDUL6NG type=del size=2949042 2024-03-04T17:50:33.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.116+0000 7fe960f568c0 1 < canceled part key=_multipart_WNXVI6K5MU3R.2~W58CXF67HEG5.0 size=1048576 2024-03-04T17:50:33.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.120+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~W58CXF67HEG5.1 size=187877 2024-03-04T17:50:33.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.124+0000 7fe960f568c0 1 < canceled multipart upload key=WNXVI6K5MU3R upload=W58CXF67HEG5 tag=7KU9SU3HWQ6F type=write size=2285029 2024-03-04T17:50:33.133 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.132+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~TICINGTXG8C6.0 size=1048576 2024-03-04T17:50:33.136 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.136+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~TICINGTXG8C6.1 size=1048576 2024-03-04T17:50:33.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.136+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~TICINGTXG8C6.2 size=315926 2024-03-04T17:50:33.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.136+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=TICINGTXG8C6 tag=7YRNN9NE88NF type=write size=3461654 2024-03-04T17:50:33.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.144+0000 7fe960f568c0 1 < suggested update operation key=GZ4RRIR5CC26 tag=9J42XL4ZMSYZ type=del size=1502756 2024-03-04T17:50:33.150 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.148+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=9T2PCKZ7PV1A type=del size=224805 2024-03-04T17:50:33.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.152+0000 7fe960f568c0 1 < canceled part key=_multipart_QGLN27WH1N2O.2~SBKUH5QE31HS.0 size=1048576 2024-03-04T17:50:33.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.152+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~SBKUH5QE31HS.1 size=518845 2024-03-04T17:50:33.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.160+0000 7fe960f568c0 1 < canceled multipart upload key=QGLN27WH1N2O upload=SBKUH5QE31HS tag=2BRRMA7B4SX2 type=write size=2615997 2024-03-04T17:50:33.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.164+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~GIUR1DYOGJ69.0 size=1048576 2024-03-04T17:50:33.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.168+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~GIUR1DYOGJ69.1 size=1048576 2024-03-04T17:50:33.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.172+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~GIUR1DYOGJ69.2 size=382203 2024-03-04T17:50:33.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.172+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=GIUR1DYOGJ69 tag=FC3OP3FX52HI type=write size=3527931 2024-03-04T17:50:33.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.180+0000 7fe960f568c0 1 < suggested update operation key=KYSLL05AUHAC tag=YY5TAZMFVUIL type=del size=1653470 2024-03-04T17:50:33.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.184+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~2E8BFSV21WXC.0 size=1048576 2024-03-04T17:50:33.189 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.188+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~2E8BFSV21WXC.1 size=44124 2024-03-04T17:50:33.190 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.188+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=2E8BFSV21WXC tag=BVEY2FUFG9YO type=write size=2141276 2024-03-04T17:50:33.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.192+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~X1GPAAVWRLJM.0 size=1048576 2024-03-04T17:50:33.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.192+0000 7fe960f568c0 1 < canceled part key=_multipart_78N8HHC4O4CG.2~X1GPAAVWRLJM.1 size=1048576 2024-03-04T17:50:33.196 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.196+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~X1GPAAVWRLJM.2 size=85017 2024-03-04T17:50:33.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.200+0000 7fe960f568c0 1 < canceled multipart upload key=78N8HHC4O4CG upload=X1GPAAVWRLJM tag=0ZSDFKSMG45Z type=write size=3230745 2024-03-04T17:50:33.206 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.204+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=RESJ6HY6T2EB type=del size=2849521 2024-03-04T17:50:33.207 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.208+0000 7fe960f568c0 1 < canceled part key=_multipart_2KZ4IFW54C5B.2~7XZJEXZLTUXF.0 size=1048576 2024-03-04T17:50:33.207 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.208+0000 7fe960f568c0 1 < canceled part key=_multipart_2KZ4IFW54C5B.2~7XZJEXZLTUXF.1 size=499465 2024-03-04T17:50:33.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.212+0000 7fe960f568c0 1 < canceled multipart upload key=2KZ4IFW54C5B upload=7XZJEXZLTUXF tag=IIX2G8QZ089K type=write size=2596617 2024-03-04T17:50:33.217 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.216+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=39QX4ILYTH37 type=del size=4101666 2024-03-04T17:50:33.221 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.220+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=YLKWRS6AC7GU type=del 2024-03-04T17:50:33.226 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.224+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=MD83I81767F9 type=del size=1139152 2024-03-04T17:50:33.231 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.228+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~JSMOQVPYJCGM.0 size=1048576 2024-03-04T17:50:33.234 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.232+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~JSMOQVPYJCGM.1 size=1048576 2024-03-04T17:50:33.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.236+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~JSMOQVPYJCGM.2 size=1041052 2024-03-04T17:50:33.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.240+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=JSMOQVPYJCGM tag=6JQT42W2ODKD type=write size=4186780 2024-03-04T17:50:33.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.248+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~XD6B55P12IVK.0 size=587085 2024-03-04T17:50:33.250 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.248+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=XD6B55P12IVK tag=G03K44HLPKHF type=write size=1635661 2024-03-04T17:50:33.254 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.252+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~N53KE57QPVYT.0 size=1048576 2024-03-04T17:50:33.257 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.256+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~N53KE57QPVYT.1 size=911494 2024-03-04T17:50:33.269 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.268+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=N53KE57QPVYT tag=E0R01I015GAB type=write size=3008646 2024-03-04T17:50:33.276 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.276+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~YNXA286P0T0T.0 size=1048576 2024-03-04T17:50:33.279 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.280+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~YNXA286P0T0T.1 size=1048576 2024-03-04T17:50:33.282 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.280+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~YNXA286P0T0T.2 size=549735 2024-03-04T17:50:33.282 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.280+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=YNXA286P0T0T tag=E3488BR9PS7I type=write size=3695463 2024-03-04T17:50:33.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.288+0000 7fe960f568c0 1 < suggested update operation key=QGLN27WH1N2O tag=Q381AI9VELJP type=del size=3294165 2024-03-04T17:50:33.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.292+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=Z86MBKXOS79S type=del size=1129924 2024-03-04T17:50:33.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.296+0000 7fe960f568c0 -1 < failed to complete operation key=KYSLL05AUHAC tag=02J9K5632HTR type=write size=893289: (22) Invalid argument 2024-03-04T17:50:33.304 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.304+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~U6UU4JA3QPV5.0 size=364104 2024-03-04T17:50:33.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.304+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=U6UU4JA3QPV5 tag=3QKPPYE39EUZ type=write size=1412680 2024-03-04T17:50:33.308 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.308+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~OA3233YRRDS7.0 size=1048576 2024-03-04T17:50:33.311 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.312+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~OA3233YRRDS7.1 size=267616 2024-03-04T17:50:33.317 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.316+0000 7fe960f568c0 1 < completed multipart upload key=XSQ3GE2UHMZG upload=OA3233YRRDS7 tag=UG8NQAM8AK00 type=write size=2364768 2024-03-04T17:50:33.324 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.324+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~WCW03UZXM3MU.0 size=305528 2024-03-04T17:50:33.324 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.324+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=WCW03UZXM3MU tag=IPNHN2BV3MSE type=write size=1354104 2024-03-04T17:50:33.327 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.328+0000 7fe960f568c0 -1 < failed to complete operation key=KYSLL05AUHAC tag=MIRKMD5XOD7L type=write size=773749: (22) Invalid argument 2024-03-04T17:50:33.334 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.332+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~OK0F33NTCJDK.0 size=1048576 2024-03-04T17:50:33.337 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.336+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~OK0F33NTCJDK.1 size=109631 2024-03-04T17:50:33.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.336+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=OK0F33NTCJDK tag=TPRDDXTQC9A0 type=write size=2206783 2024-03-04T17:50:33.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.340+0000 7fe960f568c0 1 < completed write operation key=XSQ3GE2UHMZG tag=L0LA9F9RG528 type=write size=624704 2024-03-04T17:50:33.350 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.348+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~KQ9ZKETPRPGY.0 size=1048576 2024-03-04T17:50:33.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.352+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~KQ9ZKETPRPGY.1 size=1048576 2024-03-04T17:50:33.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.356+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~KQ9ZKETPRPGY.2 size=947710 2024-03-04T17:50:33.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.356+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=KQ9ZKETPRPGY tag=YMR9C4KL7SOH type=write size=4093438 2024-03-04T17:50:33.361 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.360+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~I0S4VAKUWAX4.0 size=773988 2024-03-04T17:50:33.366 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.364+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=I0S4VAKUWAX4 tag=GL9DRLEL91ST type=write size=1822564 2024-03-04T17:50:33.370 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.368+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=KTNT5Z4BVUXR type=del size=3327801 2024-03-04T17:50:33.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.372+0000 7fe960f568c0 1 < completed delete operation key=UEVMIPQ9FR81 tag=VQTB5CDUL6NG type=del 2024-03-04T17:50:33.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.376+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=5RF7NWKO67W6 type=write size=644875 2024-03-04T17:50:33.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.376+0000 7fe960f568c0 1 < canceled part key=_multipart_R6ZDVOUE9NXJ.2~TICINGTXG8C6.0 size=1048576 2024-03-04T17:50:33.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.380+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~TICINGTXG8C6.1 size=1048576 2024-03-04T17:50:33.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.384+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~TICINGTXG8C6.2 size=315926 2024-03-04T17:50:33.391 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.388+0000 7fe960f568c0 1 < canceled multipart upload key=R6ZDVOUE9NXJ upload=TICINGTXG8C6 tag=7YRNN9NE88NF type=write size=3461654 2024-03-04T17:50:33.394 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.392+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=DPHCA1UFOIQS type=del size=1092585 2024-03-04T17:50:33.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.396+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=9T2PCKZ7PV1A type=del 2024-03-04T17:50:33.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.400+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=DJ3OE66VOW10 type=del size=3595942 2024-03-04T17:50:33.406 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.404+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~GIUR1DYOGJ69.0 size=1048576 2024-03-04T17:50:33.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.408+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~GIUR1DYOGJ69.1 size=1048576 2024-03-04T17:50:33.412 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.412+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~GIUR1DYOGJ69.2 size=382203 2024-03-04T17:50:33.418 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.416+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=GIUR1DYOGJ69 tag=FC3OP3FX52HI type=write size=3527931 2024-03-04T17:50:33.422 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.420+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=KXVHGH4OANJ0 type=write size=829599 2024-03-04T17:50:33.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.424+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~2E8BFSV21WXC.0 size=1048576 2024-03-04T17:50:33.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.424+0000 7fe960f568c0 1 < canceled part key=_multipart_UEVMIPQ9FR81.2~2E8BFSV21WXC.1 size=44124 2024-03-04T17:50:33.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.432+0000 7fe960f568c0 1 < canceled multipart upload key=UEVMIPQ9FR81 upload=2E8BFSV21WXC tag=BVEY2FUFG9YO type=write size=2141276 2024-03-04T17:50:33.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.440+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~WPIYAZEOPSYU.0 size=898485 2024-03-04T17:50:33.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.440+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=WPIYAZEOPSYU tag=9N32MEQAHI3H type=write size=1947061 2024-03-04T17:50:33.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.444+0000 7fe960f568c0 -1 < failed to complete operation key=QGLN27WH1N2O tag=RESJ6HY6T2EB type=del size=2849521: (22) Invalid argument 2024-03-04T17:50:33.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.448+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~6HYDWIAQL4ZM.0 size=1048576 2024-03-04T17:50:33.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.452+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~6HYDWIAQL4ZM.1 size=1048576 2024-03-04T17:50:33.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.456+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~6HYDWIAQL4ZM.2 size=322687 2024-03-04T17:50:33.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.456+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=6HYDWIAQL4ZM tag=IHNJJKVJOWYG type=write size=3468415 2024-03-04T17:50:33.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.464+0000 7fe960f568c0 1 < suggested update operation key=3YTAYUQLLUKY tag=39QX4ILYTH37 type=del size=4101666 2024-03-04T17:50:33.468 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.468+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=2T5S3YLZFZ2I type=del size=262830 2024-03-04T17:50:33.476 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.476+0000 7fe960f568c0 1 < suggested update operation key=U2G6YEVD5506 tag=MD83I81767F9 type=del size=1139152 2024-03-04T17:50:33.480 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.480+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=G6FNMO4YOWVZ type=del size=2725428 2024-03-04T17:50:33.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.484+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~XD6B55P12IVK.0 size=587085 2024-03-04T17:50:33.490 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.488+0000 7fe960f568c0 1 < completed multipart upload key=2MKQA5JYWU8L upload=XD6B55P12IVK tag=G03K44HLPKHF type=write size=1635661 2024-03-04T17:50:33.494 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.492+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=7JTF0ZTSGAAR type=del size=3507960 2024-03-04T17:50:33.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.496+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~YNXA286P0T0T.0 size=1048576 2024-03-04T17:50:33.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.500+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~YNXA286P0T0T.1 size=1048576 2024-03-04T17:50:33.504 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.504+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~YNXA286P0T0T.2 size=549735 2024-03-04T17:50:33.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.508+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=YNXA286P0T0T tag=E3488BR9PS7I type=write size=3695463 2024-03-04T17:50:33.514 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.512+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=LCEZB6S3R2OL type=del size=821272 2024-03-04T17:50:33.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.516+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=Z86MBKXOS79S type=del 2024-03-04T17:50:33.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.520+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=02X50GSJR8K9 type=del size=1413306 2024-03-04T17:50:33.527 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.524+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~U6UU4JA3QPV5.0 size=364104 2024-03-04T17:50:33.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.532+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=U6UU4JA3QPV5 tag=3QKPPYE39EUZ type=write size=1412680 2024-03-04T17:50:33.540 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.540+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~E8QDXQUWV8FB.0 size=1048576 2024-03-04T17:50:33.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.540+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~E8QDXQUWV8FB.1 size=1048576 2024-03-04T17:50:33.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.544+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~E8QDXQUWV8FB.2 size=369544 2024-03-04T17:50:33.546 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.544+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=E8QDXQUWV8FB tag=WB8ZNWW71953 type=write size=3515272 2024-03-04T17:50:33.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.548+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~WCW03UZXM3MU.0 size=305528 2024-03-04T17:50:33.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.556+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=WCW03UZXM3MU tag=IPNHN2BV3MSE type=write size=1354104 2024-03-04T17:50:33.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.560+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=701BZ8SXVQZX type=del size=4150615 2024-03-04T17:50:33.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.564+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~OK0F33NTCJDK.0 size=1048576 2024-03-04T17:50:33.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.564+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~OK0F33NTCJDK.1 size=109631 2024-03-04T17:50:33.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.572+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=OK0F33NTCJDK tag=TPRDDXTQC9A0 type=write size=2206783 2024-03-04T17:50:33.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.580+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~EDOWHQL3AZIU.0 size=1048576 2024-03-04T17:50:33.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.580+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~EDOWHQL3AZIU.1 size=1048576 2024-03-04T17:50:33.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.584+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~EDOWHQL3AZIU.2 size=428693 2024-03-04T17:50:33.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.584+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=EDOWHQL3AZIU tag=YGCP306XYXK2 type=write size=3574421 2024-03-04T17:50:33.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.584+0000 7fe960f568c0 1 < canceled part key=_multipart_768X2XKWJW1S.2~KQ9ZKETPRPGY.0 size=1048576 2024-03-04T17:50:33.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.588+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~KQ9ZKETPRPGY.1 size=1048576 2024-03-04T17:50:33.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.592+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~KQ9ZKETPRPGY.2 size=947710 2024-03-04T17:50:33.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.596+0000 7fe960f568c0 1 < canceled multipart upload key=768X2XKWJW1S upload=KQ9ZKETPRPGY tag=YMR9C4KL7SOH type=write size=4093438 2024-03-04T17:50:33.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.600+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=TL6MJ2X8AC4J type=write size=975013 2024-03-04T17:50:33.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.604+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=KTNT5Z4BVUXR type=del 2024-03-04T17:50:33.610 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.608+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=JF9E59TKD246 type=del size=1198484 2024-03-04T17:50:33.614 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.612+0000 7fe960f568c0 1 < canceled operation key=JT02WL9JYTNM tag=5RF7NWKO67W6 type=write size=644875 2024-03-04T17:50:33.621 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.620+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=EQ6I62I9QI47 type=write size=1036482 2024-03-04T17:50:33.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.628+0000 7fe960f568c0 1 < suggested update operation key=WNXVI6K5MU3R tag=DPHCA1UFOIQS type=del size=1092585 2024-03-04T17:50:33.633 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.632+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=TIRTW9IBWSS7 type=del size=4103280 2024-03-04T17:50:33.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.640+0000 7fe960f568c0 1 < suggested update operation key=NF4OIJWJ83DE tag=DJ3OE66VOW10 type=del size=3595942 2024-03-04T17:50:33.648 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.648+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~Q4V7VISC1W02.0 size=1048576 2024-03-04T17:50:33.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.648+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~Q4V7VISC1W02.1 size=1048576 2024-03-04T17:50:33.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.652+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~Q4V7VISC1W02.2 size=703842 2024-03-04T17:50:33.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.652+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=Q4V7VISC1W02 tag=C75N9ZUUKBPB type=write size=3849570 2024-03-04T17:50:33.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.656+0000 7fe960f568c0 1 < completed write operation key=2FUTMG0SSXBY tag=KXVHGH4OANJ0 type=write size=829599 2024-03-04T17:50:33.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.668+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~PBLJ8Q32T241.0 size=416039 2024-03-04T17:50:33.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.668+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=PBLJ8Q32T241 tag=Y9BXX30C8PJZ type=write size=1464615 2024-03-04T17:50:33.668 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.668+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~WPIYAZEOPSYU.0 size=898485 2024-03-04T17:50:33.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.672+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=WPIYAZEOPSYU tag=9N32MEQAHI3H type=write size=1947061 2024-03-04T17:50:33.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.676+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=TJI3X1M9QRVM type=write size=277385 2024-03-04T17:50:33.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.680+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~6HYDWIAQL4ZM.0 size=1048576 2024-03-04T17:50:33.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.680+0000 7fe960f568c0 1 < canceled part key=_multipart_3YTAYUQLLUKY.2~6HYDWIAQL4ZM.1 size=1048576 2024-03-04T17:50:33.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.684+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~6HYDWIAQL4ZM.2 size=322687 2024-03-04T17:50:33.692 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.692+0000 7fe960f568c0 1 < canceled multipart upload key=3YTAYUQLLUKY upload=6HYDWIAQL4ZM tag=IHNJJKVJOWYG type=write size=3468415 2024-03-04T17:50:33.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.696+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~7OSU5GIM4XHY.0 size=1048576 2024-03-04T17:50:33.701 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.700+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~7OSU5GIM4XHY.1 size=999883 2024-03-04T17:50:33.701 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.700+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=7OSU5GIM4XHY tag=56OP0IMAWPTS type=write size=3097035 2024-03-04T17:50:33.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.704+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=2T5S3YLZFZ2I type=del 2024-03-04T17:50:33.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.708+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=ATH5BNN6M855 type=write size=591910 2024-03-04T17:50:33.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.716+0000 7fe960f568c0 1 < suggested remove operation key=3X4719GHK3BI tag=G6FNMO4YOWVZ type=del 2024-03-04T17:50:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.724+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~1QO4K9GC2W2T.0 size=268051 2024-03-04T17:50:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.724+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=1QO4K9GC2W2T tag=9DQJGDMPR9XO type=write size=1316627 2024-03-04T17:50:33.728 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.728+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=7JTF0ZTSGAAR type=del 2024-03-04T17:50:33.735 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.736+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~4M4JCGGCYP6R.0 size=1048576 2024-03-04T17:50:33.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.736+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~4M4JCGGCYP6R.1 size=434507 2024-03-04T17:50:33.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.736+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=4M4JCGGCYP6R tag=9JSI21EWMFON type=write size=2531659 2024-03-04T17:50:33.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.740+0000 7fe960f568c0 -1 < failed to complete operation key=NF4OIJWJ83DE tag=LCEZB6S3R2OL type=del size=821272: (22) Invalid argument 2024-03-04T17:50:33.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.744+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=GTOJFJLM0QI3 type=write size=44202 2024-03-04T17:50:33.749 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.748+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=02X50GSJR8K9 type=del 2024-03-04T17:50:33.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.752+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=T2AJXB3LAOUS type=del size=1245602 2024-03-04T17:50:33.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.756+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~E8QDXQUWV8FB.0 size=1048576 2024-03-04T17:50:33.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.760+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~E8QDXQUWV8FB.1 size=1048576 2024-03-04T17:50:33.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.760+0000 7fe960f568c0 1 < canceled part key=_multipart_78N8HHC4O4CG.2~E8QDXQUWV8FB.2 size=369544 2024-03-04T17:50:33.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.768+0000 7fe960f568c0 1 < canceled multipart upload key=78N8HHC4O4CG upload=E8QDXQUWV8FB tag=WB8ZNWW71953 type=write size=3515272 2024-03-04T17:50:33.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.772+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~HSRP2GC15W9B.0 size=1048576 2024-03-04T17:50:33.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.776+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~HSRP2GC15W9B.1 size=423838 2024-03-04T17:50:33.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.776+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=HSRP2GC15W9B tag=9EYZM9ARKI5T type=write size=2520990 2024-03-04T17:50:33.785 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.784+0000 7fe960f568c0 1 < suggested update operation key=7Z7BPZ8XGBHZ tag=701BZ8SXVQZX type=del size=4150615 2024-03-04T17:50:33.789 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.788+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=RT2CK2K9XJC0 type=del size=2224336 2024-03-04T17:50:33.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.792+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~EDOWHQL3AZIU.0 size=1048576 2024-03-04T17:50:33.797 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.796+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~EDOWHQL3AZIU.1 size=1048576 2024-03-04T17:50:33.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.800+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~EDOWHQL3AZIU.2 size=428693 2024-03-04T17:50:33.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.804+0000 7fe960f568c0 1 < completed multipart upload key=U2G6YEVD5506 upload=EDOWHQL3AZIU tag=YGCP306XYXK2 type=write size=3574421 2024-03-04T17:50:33.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.812+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~CFOWH1JJY0EF.0 size=373113 2024-03-04T17:50:33.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.812+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=CFOWH1JJY0EF tag=KRSE8WWZU3HW type=write size=1421689 2024-03-04T17:50:33.817 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.816+0000 7fe960f568c0 1 < completed write operation key=KYSLL05AUHAC tag=TL6MJ2X8AC4J type=write size=975013 2024-03-04T17:50:33.820 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.820+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=F59BGLYODQXH type=del size=2529636 2024-03-04T17:50:33.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.824+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=JF9E59TKD246 type=del 2024-03-04T17:50:33.828 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.828+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=S021M2UF6B54 type=del size=3958121 2024-03-04T17:50:33.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.832+0000 7fe960f568c0 -1 < failed to complete operation key=7Z7BPZ8XGBHZ tag=EQ6I62I9QI47 type=write size=1036482: (22) Invalid argument 2024-03-04T17:50:33.835 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.832+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=XLYRCUHASIWJ type=del size=3818952 2024-03-04T17:50:33.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.836+0000 7fe960f568c0 1 < canceled operation key=ASC735ZCBLCG tag=TIRTW9IBWSS7 type=del size=4103280 2024-03-04T17:50:33.845 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.844+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~VB4CL2FFSVPG.0 size=1048576 2024-03-04T17:50:33.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.848+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~VB4CL2FFSVPG.1 size=151595 2024-03-04T17:50:33.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.848+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=VB4CL2FFSVPG tag=28ZNHQX736HG type=write size=2248747 2024-03-04T17:50:33.852 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.852+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~Q4V7VISC1W02.0 size=1048576 2024-03-04T17:50:33.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.856+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~Q4V7VISC1W02.1 size=1048576 2024-03-04T17:50:33.859 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.856+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~Q4V7VISC1W02.2 size=703842 2024-03-04T17:50:33.864 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.864+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=Q4V7VISC1W02 tag=C75N9ZUUKBPB type=write size=3849570 2024-03-04T17:50:33.868 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.868+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=V5H5FY89ODNX type=del size=2913374 2024-03-04T17:50:33.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.872+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~PBLJ8Q32T241.0 size=416039 2024-03-04T17:50:33.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.876+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=PBLJ8Q32T241 tag=Y9BXX30C8PJZ type=write size=1464615 2024-03-04T17:50:33.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.880+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=1N3YCF44812G type=del size=1243846 2024-03-04T17:50:33.885 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.884+0000 7fe960f568c0 1 < completed write operation key=2MKQA5JYWU8L tag=TJI3X1M9QRVM type=write size=277385 2024-03-04T17:50:33.892 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.892+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~ZNLYJBQ1HZOX.0 size=1048576 2024-03-04T17:50:33.894 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.892+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~ZNLYJBQ1HZOX.1 size=1048576 2024-03-04T17:50:33.897 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.896+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~ZNLYJBQ1HZOX.2 size=816990 2024-03-04T17:50:33.897 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.896+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=ZNLYJBQ1HZOX tag=0EGX6P9RE7Y0 type=write size=3962718 2024-03-04T17:50:33.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.900+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~7OSU5GIM4XHY.0 size=1048576 2024-03-04T17:50:33.904 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.904+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~7OSU5GIM4XHY.1 size=999883 2024-03-04T17:50:33.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.908+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=7OSU5GIM4XHY tag=56OP0IMAWPTS type=write size=3097035 2024-03-04T17:50:33.917 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.916+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~QADM3L773R26.0 size=1048576 2024-03-04T17:50:33.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.920+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~QADM3L773R26.1 size=868382 2024-03-04T17:50:33.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.920+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=QADM3L773R26 tag=U1G8HL9F473T type=write size=2965534 2024-03-04T17:50:33.924 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.924+0000 7fe960f568c0 1 < completed write operation key=XSQ3GE2UHMZG tag=ATH5BNN6M855 type=write size=591910 2024-03-04T17:50:33.928 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.928+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=SKWGHYK9YF7O type=del size=2371851 2024-03-04T17:50:33.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.932+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~1QO4K9GC2W2T.0 size=268051 2024-03-04T17:50:33.938 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.936+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=1QO4K9GC2W2T tag=9DQJGDMPR9XO type=write size=1316627 2024-03-04T17:50:33.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.940+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=3OMI7A223FZX type=del size=3344066 2024-03-04T17:50:33.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.944+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~4M4JCGGCYP6R.0 size=1048576 2024-03-04T17:50:33.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.948+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~4M4JCGGCYP6R.1 size=434507 2024-03-04T17:50:33.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.952+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=4M4JCGGCYP6R tag=9JSI21EWMFON type=write size=2531659 2024-03-04T17:50:33.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.956+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=UVXUDJC1XQJP type=del size=3062616 2024-03-04T17:50:33.965 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.964+0000 7fe960f568c0 1 < suggested remove operation key=3X4719GHK3BI tag=GTOJFJLM0QI3 type=write 2024-03-04T17:50:33.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.972+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=JEPN6CC7DZNZ type=del size=1471451 2024-03-04T17:50:33.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.976+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=T2AJXB3LAOUS type=del 2024-03-04T17:50:33.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.980+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=GNU6KQ4XNT2J type=del size=659590 2024-03-04T17:50:33.985 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.984+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~HSRP2GC15W9B.0 size=1048576 2024-03-04T17:50:33.988 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.988+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~HSRP2GC15W9B.1 size=423838 2024-03-04T17:50:33.994 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:33.992+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=HSRP2GC15W9B tag=9EYZM9ARKI5T type=write size=2520990 2024-03-04T17:50:34.001 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.000+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~I4V4A1PAII0J.0 size=1048576 2024-03-04T17:50:34.004 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.004+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~I4V4A1PAII0J.1 size=1048576 2024-03-04T17:50:34.007 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.004+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~I4V4A1PAII0J.2 size=902465 2024-03-04T17:50:34.007 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.004+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=I4V4A1PAII0J tag=NSXEVCJL96G7 type=write size=4048193 2024-03-04T17:50:34.010 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.008+0000 7fe960f568c0 1 < completed delete operation key=U2G6YEVD5506 tag=RT2CK2K9XJC0 type=del 2024-03-04T17:50:34.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.012+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=AKSSTDPCU9TA type=del size=1569397 2024-03-04T17:50:34.018 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.016+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~CFOWH1JJY0EF.0 size=373113 2024-03-04T17:50:34.024 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.024+0000 7fe960f568c0 1 < completed multipart upload key=XSQ3GE2UHMZG upload=CFOWH1JJY0EF tag=KRSE8WWZU3HW type=write size=1421689 2024-03-04T17:50:34.028 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.028+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=IQKVQ0KX4UJD type=del size=653967 2024-03-04T17:50:34.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.032+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=F59BGLYODQXH type=del 2024-03-04T17:50:34.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.036+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=XQFNNJ2RCJX9 type=del size=2845657 2024-03-04T17:50:34.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.040+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=S021M2UF6B54 type=del 2024-03-04T17:50:34.043 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.044+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=ZZ00D3RJT85V type=del size=3404201 2024-03-04T17:50:34.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.044+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=XLYRCUHASIWJ type=del 2024-03-04T17:50:34.053 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.052+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~R3LSW7LI2EDW.0 size=838926 2024-03-04T17:50:34.053 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.052+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=R3LSW7LI2EDW tag=MPHB8R1KVZV8 type=write size=1887502 2024-03-04T17:50:34.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.056+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~VB4CL2FFSVPG.0 size=1048576 2024-03-04T17:50:34.060 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.060+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~VB4CL2FFSVPG.1 size=151595 2024-03-04T17:50:34.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.064+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=VB4CL2FFSVPG tag=28ZNHQX736HG type=write size=2248747 2024-03-04T17:50:34.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.068+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=LZACY8SGDY9H type=del size=216981 2024-03-04T17:50:34.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.072+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=V5H5FY89ODNX type=del 2024-03-04T17:50:34.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.076+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=GTXGKM5IQP4P type=write size=289556 2024-03-04T17:50:34.080 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.080+0000 7fe960f568c0 1 < canceled operation key=JT02WL9JYTNM tag=1N3YCF44812G type=del size=1243846 2024-03-04T17:50:34.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.084+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=KI6WBHAXHKAS type=del size=518240 2024-03-04T17:50:34.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.088+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~ZNLYJBQ1HZOX.0 size=1048576 2024-03-04T17:50:34.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.088+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~ZNLYJBQ1HZOX.1 size=1048576 2024-03-04T17:50:34.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.092+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~ZNLYJBQ1HZOX.2 size=816990 2024-03-04T17:50:34.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.100+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=ZNLYJBQ1HZOX tag=0EGX6P9RE7Y0 type=write size=3962718 2024-03-04T17:50:34.103 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.104+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=ZIWOWLNM737Q type=del size=1093585 2024-03-04T17:50:34.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.108+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~QADM3L773R26.0 size=1048576 2024-03-04T17:50:34.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.108+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~QADM3L773R26.1 size=868382 2024-03-04T17:50:34.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.116+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=QADM3L773R26 tag=U1G8HL9F473T type=write size=2965534 2024-03-04T17:50:34.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.124+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~X0D0AT32IQDD.0 size=1048576 2024-03-04T17:50:34.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.124+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~X0D0AT32IQDD.1 size=664866 2024-03-04T17:50:34.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.124+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=X0D0AT32IQDD tag=PDGNU8E81ELI type=write size=2762018 2024-03-04T17:50:34.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.128+0000 7fe960f568c0 1 < canceled operation key=QGLN27WH1N2O tag=SKWGHYK9YF7O type=del size=2371851 2024-03-04T17:50:34.136 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.136+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~BYPYCOHV9BFD.0 size=1048576 2024-03-04T17:50:34.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.136+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~BYPYCOHV9BFD.1 size=431664 2024-03-04T17:50:34.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.140+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=BYPYCOHV9BFD tag=53D9HEPFG577 type=write size=2528816 2024-03-04T17:50:34.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.144+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=3OMI7A223FZX type=del 2024-03-04T17:50:34.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.144+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=36XQ85INUZFH type=write size=479363 2024-03-04T17:50:34.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.152+0000 7fe960f568c0 1 < suggested update operation key=UEVMIPQ9FR81 tag=UVXUDJC1XQJP type=del size=3062616 2024-03-04T17:50:34.159 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.156+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=U3IKXB6GZJ3Q type=del size=929742 2024-03-04T17:50:34.166 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.164+0000 7fe960f568c0 1 < suggested remove operation key=U2G6YEVD5506 tag=JEPN6CC7DZNZ type=del 2024-03-04T17:50:34.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.168+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=VHBUNV9CUNPV type=del size=2337596 2024-03-04T17:50:34.174 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.172+0000 7fe960f568c0 -1 < failed to complete operation key=UEVMIPQ9FR81 tag=GNU6KQ4XNT2J type=del size=659590: (22) Invalid argument 2024-03-04T17:50:34.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.176+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=M22LR305WI14 type=del size=2297671 2024-03-04T17:50:34.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.176+0000 7fe960f568c0 1 < canceled part key=_multipart_3YTAYUQLLUKY.2~I4V4A1PAII0J.0 size=1048576 2024-03-04T17:50:34.182 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.180+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~I4V4A1PAII0J.1 size=1048576 2024-03-04T17:50:34.185 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.184+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~I4V4A1PAII0J.2 size=902465 2024-03-04T17:50:34.191 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.192+0000 7fe960f568c0 1 < canceled multipart upload key=3YTAYUQLLUKY upload=I4V4A1PAII0J tag=NSXEVCJL96G7 type=write size=4048193 2024-03-04T17:50:34.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.192+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=FAJK7LRCS5BD type=del size=3750185 2024-03-04T17:50:34.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.196+0000 7fe960f568c0 1 < canceled operation key=RII14ZGX2EL1 tag=AKSSTDPCU9TA type=del size=1569397 2024-03-04T17:50:34.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.200+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=HW7SR3LGVO1R type=write size=687306 2024-03-04T17:50:34.206 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.204+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=IQKVQ0KX4UJD type=del 2024-03-04T17:50:34.210 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.208+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=LUTAJHJJD4DW type=del size=2662312 2024-03-04T17:50:34.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.212+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=XQFNNJ2RCJX9 type=del 2024-03-04T17:50:34.220 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.220+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~EKQAY3GJUUS5.0 size=613289 2024-03-04T17:50:34.220 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.220+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=EKQAY3GJUUS5 tag=4CFHFGDKHSGI type=write size=1661865 2024-03-04T17:50:34.228 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.228+0000 7fe960f568c0 1 < suggested remove operation key=3X4719GHK3BI tag=ZZ00D3RJT85V type=del 2024-03-04T17:50:34.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.232+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~R3FK1TTJI21L.0 size=527919 2024-03-04T17:50:34.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.232+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=R3FK1TTJI21L tag=E818XERA1C6O type=write size=1576495 2024-03-04T17:50:34.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.240+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~R3LSW7LI2EDW.0 size=838926 2024-03-04T17:50:34.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.244+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=R3LSW7LI2EDW tag=MPHB8R1KVZV8 type=write size=1887502 2024-03-04T17:50:34.253 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.252+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~OBEPTAE8VCS1.0 size=1048576 2024-03-04T17:50:34.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.256+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~OBEPTAE8VCS1.1 size=1048576 2024-03-04T17:50:34.258 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.256+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~OBEPTAE8VCS1.2 size=1041480 2024-03-04T17:50:34.258 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.256+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=OBEPTAE8VCS1 tag=XGI8TE1ZYV97 type=write size=4187208 2024-03-04T17:50:34.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.260+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=LZACY8SGDY9H type=del 2024-03-04T17:50:34.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.264+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=MY0R2C3AEY52 type=del size=552296 2024-03-04T17:50:34.269 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.268+0000 7fe960f568c0 -1 < failed to complete operation key=U2G6YEVD5506 tag=GTXGKM5IQP4P type=write size=289556: (22) Invalid argument 2024-03-04T17:50:34.273 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.272+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=LY2D8XT0KW0C type=del size=3166989 2024-03-04T17:50:34.277 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.276+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=KI6WBHAXHKAS type=del 2024-03-04T17:50:34.283 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.284+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~SYM5D7I9OB6P.0 size=972310 2024-03-04T17:50:34.283 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.284+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=SYM5D7I9OB6P tag=AZ3C2LUSP92M type=write size=2020886 2024-03-04T17:50:34.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.288+0000 7fe960f568c0 1 < canceled operation key=R6ZDVOUE9NXJ tag=ZIWOWLNM737Q type=del size=1093585 2024-03-04T17:50:34.292 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.292+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=DF5U2DKJVKUG type=write size=498795 2024-03-04T17:50:34.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.296+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~X0D0AT32IQDD.0 size=1048576 2024-03-04T17:50:34.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.296+0000 7fe960f568c0 1 < canceled part key=_multipart_JT02WL9JYTNM.2~X0D0AT32IQDD.1 size=664866 2024-03-04T17:50:34.302 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.300+0000 7fe960f568c0 1 < canceled multipart upload key=JT02WL9JYTNM upload=X0D0AT32IQDD tag=PDGNU8E81ELI type=write size=2762018 2024-03-04T17:50:34.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.304+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=R95IKTA2J8AR type=del size=1603776 2024-03-04T17:50:34.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.308+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~BYPYCOHV9BFD.0 size=1048576 2024-03-04T17:50:34.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.312+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~BYPYCOHV9BFD.1 size=431664 2024-03-04T17:50:34.318 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.316+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=BYPYCOHV9BFD tag=53D9HEPFG577 type=write size=2528816 2024-03-04T17:50:34.326 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.324+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=YE46EJMOZUNE type=del size=1865596 2024-03-04T17:50:34.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.328+0000 7fe960f568c0 1 < canceled operation key=P28N6U79ZUV2 tag=36XQ85INUZFH type=write size=479363 2024-03-04T17:50:34.334 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.332+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=HGGPFS2RF49J type=del size=1911872 2024-03-04T17:50:34.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.340+0000 7fe960f568c0 1 < suggested update operation key=8PZD01OOX3AF tag=U3IKXB6GZJ3Q type=del size=929742 2024-03-04T17:50:34.348 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.348+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~2V4U9P1NX56S.0 size=1048576 2024-03-04T17:50:34.351 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.348+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~2V4U9P1NX56S.1 size=1048576 2024-03-04T17:50:34.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.352+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~2V4U9P1NX56S.2 size=2149 2024-03-04T17:50:34.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.352+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=2V4U9P1NX56S tag=GN5OXT376XUF type=write size=3147877 2024-03-04T17:50:34.362 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.360+0000 7fe960f568c0 1 < suggested remove operation key=JJ7SM1Y73A4C tag=VHBUNV9CUNPV type=del 2024-03-04T17:50:34.369 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.368+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~D53LX0RM8UX4.0 size=1048576 2024-03-04T17:50:34.371 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.372+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~D53LX0RM8UX4.1 size=1048576 2024-03-04T17:50:34.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.372+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~D53LX0RM8UX4.2 size=893915 2024-03-04T17:50:34.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.372+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=D53LX0RM8UX4 tag=23E9XSO88VEJ type=write size=4039643 2024-03-04T17:50:34.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.376+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=M22LR305WI14 type=del 2024-03-04T17:50:34.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.380+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=BHUZN83JZXAI type=del size=1830884 2024-03-04T17:50:34.390 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.388+0000 7fe960f568c0 1 < suggested remove operation key=8PZD01OOX3AF tag=FAJK7LRCS5BD type=del 2024-03-04T17:50:34.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.392+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=HYJ82ESCEQR4 type=del size=3229571 2024-03-04T17:50:34.397 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.396+0000 7fe960f568c0 -1 < failed to complete operation key=JJ7SM1Y73A4C tag=HW7SR3LGVO1R type=write size=687306: (22) Invalid argument 2024-03-04T17:50:34.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.400+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=275EON7HU5V8 type=del size=3566160 2024-03-04T17:50:34.407 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.408+0000 7fe960f568c0 1 < canceled operation key=RII14ZGX2EL1 tag=LUTAJHJJD4DW type=del size=2662312 2024-03-04T17:50:34.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.412+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=P26BZGD4CHZT type=del size=2505333 2024-03-04T17:50:34.415 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.416+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~EKQAY3GJUUS5.0 size=613289 2024-03-04T17:50:34.422 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.420+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=EKQAY3GJUUS5 tag=4CFHFGDKHSGI type=write size=1661865 2024-03-04T17:50:34.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.424+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=WCQTYY3BE1EY type=del size=1290581 2024-03-04T17:50:34.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.428+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~R3FK1TTJI21L.0 size=527919 2024-03-04T17:50:34.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.436+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=R3FK1TTJI21L tag=E818XERA1C6O type=write size=1576495 2024-03-04T17:50:34.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.440+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=H8JPJ3YUQNLL type=del size=1837695 2024-03-04T17:50:34.444 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.444+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~OBEPTAE8VCS1.0 size=1048576 2024-03-04T17:50:34.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.444+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~OBEPTAE8VCS1.1 size=1048576 2024-03-04T17:50:34.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.448+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~OBEPTAE8VCS1.2 size=1041480 2024-03-04T17:50:34.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.456+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=OBEPTAE8VCS1 tag=XGI8TE1ZYV97 type=write size=4187208 2024-03-04T17:50:34.460 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.460+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=2IXQ2C6FUYDM type=del size=259425 2024-03-04T17:50:34.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.464+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=MY0R2C3AEY52 type=del 2024-03-04T17:50:34.467 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.468+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=GSF5NAZ9AKCP type=del size=1728802 2024-03-04T17:50:34.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.472+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=LY2D8XT0KW0C type=del 2024-03-04T17:50:34.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.476+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=2EGKBJTZS6K0 type=del size=3927774 2024-03-04T17:50:34.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.480+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~SYM5D7I9OB6P.0 size=972310 2024-03-04T17:50:34.486 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.484+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=SYM5D7I9OB6P tag=AZ3C2LUSP92M type=write size=2020886 2024-03-04T17:50:34.490 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.488+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=IQZM2JHVT3Z0 type=write size=746111 2024-03-04T17:50:34.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.492+0000 7fe960f568c0 1 < completed write operation key=JP3FRB6MD7B4 tag=DF5U2DKJVKUG type=write size=498795 2024-03-04T17:50:34.499 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.500+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=Y3T9HTNBHCQH type=del size=4068381 2024-03-04T17:50:34.504 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.504+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=R95IKTA2J8AR type=del 2024-03-04T17:50:34.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.508+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=BTSAVFC7AVVY type=del size=3662602 2024-03-04T17:50:34.512 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.512+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=YE46EJMOZUNE type=del 2024-03-04T17:50:34.518 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.516+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~8AZQ1AKWR310.0 size=428675 2024-03-04T17:50:34.518 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.516+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=8AZQ1AKWR310 tag=WS3BBWX7K53N type=write size=1477251 2024-03-04T17:50:34.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.520+0000 7fe960f568c0 -1 < failed to complete operation key=8PZD01OOX3AF tag=HGGPFS2RF49J type=del size=1911872: (22) Invalid argument 2024-03-04T17:50:34.526 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.524+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=KHG35B0W6JIX type=write size=1001355 2024-03-04T17:50:34.530 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.528+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~2V4U9P1NX56S.0 size=1048576 2024-03-04T17:50:34.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.532+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~2V4U9P1NX56S.1 size=1048576 2024-03-04T17:50:34.536 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.536+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~2V4U9P1NX56S.2 size=2149 2024-03-04T17:50:34.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.540+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=2V4U9P1NX56S tag=GN5OXT376XUF type=write size=3147877 2024-03-04T17:50:34.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.544+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=REUT52XTMT75 type=write size=595136 2024-03-04T17:50:34.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.548+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~D53LX0RM8UX4.0 size=1048576 2024-03-04T17:50:34.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.548+0000 7fe960f568c0 1 < canceled part key=_multipart_MIZRUP4MEA0J.2~D53LX0RM8UX4.1 size=1048576 2024-03-04T17:50:34.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.552+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~D53LX0RM8UX4.2 size=893915 2024-03-04T17:50:34.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.556+0000 7fe960f568c0 1 < canceled multipart upload key=MIZRUP4MEA0J upload=D53LX0RM8UX4 tag=23E9XSO88VEJ type=write size=4039643 2024-03-04T17:50:34.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.564+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~4D9FEFTVBCF3.0 size=1048576 2024-03-04T17:50:34.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.564+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~4D9FEFTVBCF3.1 size=120925 2024-03-04T17:50:34.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.564+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=4D9FEFTVBCF3 tag=UJ4PAIKKXV10 type=write size=2218077 2024-03-04T17:50:34.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.568+0000 7fe960f568c0 1 < completed delete operation key=7Z7BPZ8XGBHZ tag=BHUZN83JZXAI type=del 2024-03-04T17:50:34.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.572+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=8CT270TSB6HR type=del size=3622980 2024-03-04T17:50:34.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.580+0000 7fe960f568c0 1 < suggested remove operation key=768X2XKWJW1S tag=HYJ82ESCEQR4 type=del 2024-03-04T17:50:34.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.584+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=5P0PFSCKXAOT type=del size=3805963 2024-03-04T17:50:34.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.588+0000 7fe960f568c0 -1 < failed to complete operation key=768X2XKWJW1S tag=275EON7HU5V8 type=del size=3566160: (22) Invalid argument 2024-03-04T17:50:34.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.592+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=FYR00PNDL0AP type=del size=2337848 2024-03-04T17:50:34.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.596+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=P26BZGD4CHZT type=del 2024-03-04T17:50:34.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.604+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~0NCJH0JL65CL.0 size=1048576 2024-03-04T17:50:34.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.604+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~0NCJH0JL65CL.1 size=83100 2024-03-04T17:50:34.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.604+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=0NCJH0JL65CL tag=WN2C08NRKG0M type=write size=2180252 2024-03-04T17:50:34.610 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.608+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=WCQTYY3BE1EY type=del 2024-03-04T17:50:34.614 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.612+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=O4V85ZPABRAP type=del size=26499 2024-03-04T17:50:34.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.616+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=H8JPJ3YUQNLL type=del 2024-03-04T17:50:34.623 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.620+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=S083F8UNVIRO type=del size=3565712 2024-03-04T17:50:34.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.628+0000 7fe960f568c0 1 < suggested remove operation key=JJ7SM1Y73A4C tag=2IXQ2C6FUYDM type=del 2024-03-04T17:50:34.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.632+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=LM6U2FG8N6ST type=del size=1998063 2024-03-04T17:50:34.639 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.640+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=GSF5NAZ9AKCP type=del 2024-03-04T17:50:34.643 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.644+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=PWSNF5KKGFGW type=del size=2743720 2024-03-04T17:50:34.648 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.648+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=2EGKBJTZS6K0 type=del 2024-03-04T17:50:34.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.652+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~K9AM72O1VF36.0 size=1048576 2024-03-04T17:50:34.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.656+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~K9AM72O1VF36.1 size=720937 2024-03-04T17:50:34.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.656+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=K9AM72O1VF36 tag=ASI5MX9IDKXS type=write size=2818089 2024-03-04T17:50:34.660 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.660+0000 7fe960f568c0 1 < completed write operation key=JT02WL9JYTNM tag=IQZM2JHVT3Z0 type=write size=746111 2024-03-04T17:50:34.663 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.664+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=SOFB2PRIB8KR type=del size=1680012 2024-03-04T17:50:34.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.668+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=Y3T9HTNBHCQH type=del 2024-03-04T17:50:34.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.672+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=EA8EPM0EGPVG type=del size=111716 2024-03-04T17:50:34.675 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.672+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=BTSAVFC7AVVY type=del 2024-03-04T17:50:34.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.676+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=PNFAPIJ3NIQ6 type=del size=906202 2024-03-04T17:50:34.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.684+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~8AZQ1AKWR310.0 size=428675 2024-03-04T17:50:34.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.692+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=8AZQ1AKWR310 tag=WS3BBWX7K53N type=write size=1477251 2024-03-04T17:50:34.702 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.700+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~SD5NYHEVHR97.0 size=1048576 2024-03-04T17:50:34.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.704+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~SD5NYHEVHR97.1 size=1048576 2024-03-04T17:50:34.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.708+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~SD5NYHEVHR97.2 size=705382 2024-03-04T17:50:34.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.708+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=SD5NYHEVHR97 tag=SE6Y0K8NM7GG type=write size=3851110 2024-03-04T17:50:34.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.708+0000 7fe960f568c0 1 < completed write operation key=KYSLL05AUHAC tag=KHG35B0W6JIX type=write size=1001355 2024-03-04T17:50:34.714 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.712+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=6HPU13AE5W9W type=del size=1715586 2024-03-04T17:50:34.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.716+0000 7fe960f568c0 1 < canceled operation key=FSWBX523X9V5 tag=REUT52XTMT75 type=write size=595136 2024-03-04T17:50:34.723 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.720+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=HYO7L28BFTIM type=del size=2171772 2024-03-04T17:50:34.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.724+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~4D9FEFTVBCF3.0 size=1048576 2024-03-04T17:50:34.729 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.728+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~4D9FEFTVBCF3.1 size=120925 2024-03-04T17:50:34.735 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.736+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=4D9FEFTVBCF3 tag=UJ4PAIKKXV10 type=write size=2218077 2024-03-04T17:50:34.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.740+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~XFJNRTXCVVRU.0 size=1048576 2024-03-04T17:50:34.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.744+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~XFJNRTXCVVRU.1 size=212720 2024-03-04T17:50:34.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.744+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=XFJNRTXCVVRU tag=QO61EOUDE7YP type=write size=2309872 2024-03-04T17:50:34.749 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.748+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=8CT270TSB6HR type=del 2024-03-04T17:50:34.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.752+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=3BT5VBJ0S4MB type=write size=510931 2024-03-04T17:50:34.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.756+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=5P0PFSCKXAOT type=del 2024-03-04T17:50:34.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.760+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~V2RU6AF839TW.0 size=450468 2024-03-04T17:50:34.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.760+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=V2RU6AF839TW tag=8KO6ZSTQ126L type=write size=1499044 2024-03-04T17:50:34.767 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.764+0000 7fe960f568c0 1 < canceled operation key=2FUTMG0SSXBY tag=FYR00PNDL0AP type=del size=2337848 2024-03-04T17:50:34.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.768+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=GGNJIZFGVFG6 type=del size=3772843 2024-03-04T17:50:34.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.772+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~0NCJH0JL65CL.0 size=1048576 2024-03-04T17:50:34.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.772+0000 7fe960f568c0 1 < canceled part key=_multipart_78N8HHC4O4CG.2~0NCJH0JL65CL.1 size=83100 2024-03-04T17:50:34.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.780+0000 7fe960f568c0 1 < canceled multipart upload key=78N8HHC4O4CG upload=0NCJH0JL65CL tag=WN2C08NRKG0M type=write size=2180252 2024-03-04T17:50:34.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.784+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=X888WRYLPO4F type=del size=705457 2024-03-04T17:50:34.788 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.788+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=O4V85ZPABRAP type=del 2024-03-04T17:50:34.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.792+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~XT8N1U2N0VZJ.0 size=22599 2024-03-04T17:50:34.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.792+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=XT8N1U2N0VZJ tag=FU3I9NGCML1E type=write size=1071175 2024-03-04T17:50:34.798 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.796+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=S083F8UNVIRO type=del 2024-03-04T17:50:34.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.800+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=DELHLWXTWSWS type=write size=269295 2024-03-04T17:50:34.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.804+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=LM6U2FG8N6ST type=del 2024-03-04T17:50:34.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.808+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=SADT6LFUV52Z type=write size=968874 2024-03-04T17:50:34.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.812+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=PWSNF5KKGFGW type=del 2024-03-04T17:50:34.817 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.816+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=1MGJ8O6PBUSP type=del size=3106105 2024-03-04T17:50:34.821 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.820+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~K9AM72O1VF36.0 size=1048576 2024-03-04T17:50:34.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.824+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~K9AM72O1VF36.1 size=720937 2024-03-04T17:50:34.830 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.828+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=K9AM72O1VF36 tag=ASI5MX9IDKXS type=write size=2818089 2024-03-04T17:50:34.834 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.832+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=YB6L98A9ONLP type=write size=222955 2024-03-04T17:50:34.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.836+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=SOFB2PRIB8KR type=del 2024-03-04T17:50:34.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.840+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=WICXO6KX80YI type=del size=136320 2024-03-04T17:50:34.850 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.848+0000 7fe960f568c0 1 < suggested update operation key=JSO6WIYMPCF6 tag=EA8EPM0EGPVG type=del size=111716 2024-03-04T17:50:34.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.852+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=H4ZEHNCT00GE type=del size=1984268 2024-03-04T17:50:34.858 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.856+0000 7fe960f568c0 1 < completed delete operation key=ASC735ZCBLCG tag=PNFAPIJ3NIQ6 type=del 2024-03-04T17:50:34.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.864+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~CY6EAQ0UBHYS.0 size=25444 2024-03-04T17:50:34.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.864+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=CY6EAQ0UBHYS tag=T1UD2R66SEAU type=write size=1074020 2024-03-04T17:50:34.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.868+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~SD5NYHEVHR97.0 size=1048576 2024-03-04T17:50:34.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.872+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~SD5NYHEVHR97.1 size=1048576 2024-03-04T17:50:34.875 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.876+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~SD5NYHEVHR97.2 size=705382 2024-03-04T17:50:34.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.880+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=SD5NYHEVHR97 tag=SE6Y0K8NM7GG type=write size=3851110 2024-03-04T17:50:34.885 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.884+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=A9VUUA6995SU type=del size=1579464 2024-03-04T17:50:34.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.888+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=6HPU13AE5W9W type=del 2024-03-04T17:50:34.893 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.892+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=RQHDBRO5VMQ1 type=del size=1421255 2024-03-04T17:50:34.897 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.896+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=HYO7L28BFTIM type=del 2024-03-04T17:50:34.904 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.904+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~64GYERFYRO43.0 size=1048576 2024-03-04T17:50:34.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.904+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~64GYERFYRO43.1 size=882242 2024-03-04T17:50:34.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.904+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=64GYERFYRO43 tag=M6ZS5O9WOA6F type=write size=2979394 2024-03-04T17:50:34.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.908+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~XFJNRTXCVVRU.0 size=1048576 2024-03-04T17:50:34.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.912+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~XFJNRTXCVVRU.1 size=212720 2024-03-04T17:50:34.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.920+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=XFJNRTXCVVRU tag=QO61EOUDE7YP type=write size=2309872 2024-03-04T17:50:34.926 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.924+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~RMBZ1QZR45O7.0 size=1048576 2024-03-04T17:50:34.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.928+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~RMBZ1QZR45O7.1 size=1048576 2024-03-04T17:50:34.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.932+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~RMBZ1QZR45O7.2 size=735399 2024-03-04T17:50:34.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.932+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=RMBZ1QZR45O7 tag=MTXLEKO31DKA type=write size=3881127 2024-03-04T17:50:34.937 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.936+0000 7fe960f568c0 1 < completed write operation key=768X2XKWJW1S tag=3BT5VBJ0S4MB type=write size=510931 2024-03-04T17:50:34.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.940+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=D2BXS3G5YGR3 type=del size=1038528 2024-03-04T17:50:34.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.944+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~V2RU6AF839TW.0 size=450468 2024-03-04T17:50:34.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.948+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=V2RU6AF839TW tag=8KO6ZSTQ126L type=write size=1499044 2024-03-04T17:50:34.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.952+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=FS7OLTBV34RX type=del size=3372204 2024-03-04T17:50:34.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.956+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=GGNJIZFGVFG6 type=del 2024-03-04T17:50:34.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.960+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=3S6D6U8YLI9J type=del size=3323286 2024-03-04T17:50:34.965 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.964+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=X888WRYLPO4F type=del 2024-03-04T17:50:34.969 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.968+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=O65KNO9SU14Q type=del size=4163200 2024-03-04T17:50:34.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.972+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~XT8N1U2N0VZJ.0 size=22599 2024-03-04T17:50:34.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.980+0000 7fe960f568c0 1 < completed multipart upload key=2MKQA5JYWU8L upload=XT8N1U2N0VZJ tag=FU3I9NGCML1E type=write size=1071175 2024-03-04T17:50:34.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.984+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=BU02QQIOVUVE type=del size=1722839 2024-03-04T17:50:34.992 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.992+0000 7fe960f568c0 1 < suggested update operation key=P28N6U79ZUV2 tag=DELHLWXTWSWS type=write size=269295 2024-03-04T17:50:34.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:34.996+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=MMH85A4Q9M92 type=del size=238770 2024-03-04T17:50:35.000 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.000+0000 7fe960f568c0 1 < completed write operation key=QGLN27WH1N2O tag=SADT6LFUV52Z type=write size=968874 2024-03-04T17:50:35.004 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.004+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=2CYUJRYX0HA0 type=write size=658720 2024-03-04T17:50:35.008 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.008+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=1MGJ8O6PBUSP type=del 2024-03-04T17:50:35.016 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.020+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~BVB1UBSU32E7.0 size=1048576 2024-03-04T17:50:35.019 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.020+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~BVB1UBSU32E7.1 size=478196 2024-03-04T17:50:35.019 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.020+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=BVB1UBSU32E7 tag=ND2T0KPZT9R2 type=write size=2575348 2024-03-04T17:50:35.022 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.020+0000 7fe960f568c0 1 < completed write operation key=3YTAYUQLLUKY tag=YB6L98A9ONLP type=write size=222955 2024-03-04T17:50:35.027 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.024+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=8IU9AU7MG3T7 type=del size=2983692 2024-03-04T17:50:35.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.028+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=WICXO6KX80YI type=del 2024-03-04T17:50:35.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.032+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=7I6RAHA305E2 type=del size=2251974 2024-03-04T17:50:35.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.040+0000 7fe960f568c0 1 < suggested remove operation key=UEVMIPQ9FR81 tag=H4ZEHNCT00GE type=del 2024-03-04T17:50:35.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.044+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=RP71C39H2J5L type=del size=2317195 2024-03-04T17:50:35.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.048+0000 7fe960f568c0 1 < canceled part key=_multipart_QFF67JIW8Y8E.2~CY6EAQ0UBHYS.0 size=25444 2024-03-04T17:50:35.059 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.056+0000 7fe960f568c0 1 < canceled multipart upload key=QFF67JIW8Y8E upload=CY6EAQ0UBHYS tag=T1UD2R66SEAU type=write size=1074020 2024-03-04T17:50:35.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.064+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=F08WFZC3V1HB type=del size=776377 2024-03-04T17:50:35.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.068+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=A9VUUA6995SU type=del 2024-03-04T17:50:35.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.076+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=39APZCJG05GJ type=del size=274308 2024-03-04T17:50:35.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.076+0000 7fe960f568c0 1 < canceled operation key=KYSLL05AUHAC tag=RQHDBRO5VMQ1 type=del size=1421255 2024-03-04T17:50:35.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.080+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=XE53DXCBZD4F type=del size=4187322 2024-03-04T17:50:35.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.080+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~64GYERFYRO43.0 size=1048576 2024-03-04T17:50:35.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.084+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~64GYERFYRO43.1 size=882242 2024-03-04T17:50:35.091 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.092+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=64GYERFYRO43 tag=M6ZS5O9WOA6F type=write size=2979394 2024-03-04T17:50:35.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.096+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~8F1MAHTX7RI5.0 size=1048576 2024-03-04T17:50:35.101 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.100+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~8F1MAHTX7RI5.1 size=719951 2024-03-04T17:50:35.101 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.100+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=8F1MAHTX7RI5 tag=K5H5NZ8RU285 type=write size=2817103 2024-03-04T17:50:35.105 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.104+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~RMBZ1QZR45O7.0 size=1048576 2024-03-04T17:50:35.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.108+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~RMBZ1QZR45O7.1 size=1048576 2024-03-04T17:50:35.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.108+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~RMBZ1QZR45O7.2 size=735399 2024-03-04T17:50:35.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.116+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=RMBZ1QZR45O7 tag=MTXLEKO31DKA type=write size=3881127 2024-03-04T17:50:35.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.120+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=2U09OZY6BELI type=del size=2051523 2024-03-04T17:50:35.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.124+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=D2BXS3G5YGR3 type=del 2024-03-04T17:50:35.129 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.128+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=X4QSYPGC5YGM type=del size=2980124 2024-03-04T17:50:35.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.132+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=FS7OLTBV34RX type=del 2024-03-04T17:50:35.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.136+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~0QCR48Q91ZG7.0 size=498692 2024-03-04T17:50:35.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.136+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=0QCR48Q91ZG7 tag=OGX63JUBHSWU type=write size=1547268 2024-03-04T17:50:35.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.140+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=3S6D6U8YLI9J type=del 2024-03-04T17:50:35.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.144+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=GUVQ5JL92KLT type=del size=213454 2024-03-04T17:50:35.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.148+0000 7fe960f568c0 1 < completed delete operation key=ASC735ZCBLCG tag=O65KNO9SU14Q type=del 2024-03-04T17:50:35.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.156+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~XDHFJ951T98X.0 size=1048576 2024-03-04T17:50:35.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.160+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~XDHFJ951T98X.1 size=157244 2024-03-04T17:50:35.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.160+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=XDHFJ951T98X tag=T35TBUW3NX45 type=write size=2254396 2024-03-04T17:50:35.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.164+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=BU02QQIOVUVE type=del 2024-03-04T17:50:35.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.172+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~2OQSW0XVVJW6.0 size=1048576 2024-03-04T17:50:35.174 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.172+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~2OQSW0XVVJW6.1 size=1048576 2024-03-04T17:50:35.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.176+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~2OQSW0XVVJW6.2 size=11352 2024-03-04T17:50:35.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.176+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=2OQSW0XVVJW6 tag=VG7LD9P1RERY type=write size=3157080 2024-03-04T17:50:35.181 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.180+0000 7fe960f568c0 1 < canceled operation key=FSWBX523X9V5 tag=MMH85A4Q9M92 type=del size=238770 2024-03-04T17:50:35.185 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.184+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=C3SN9289WNPF type=del size=1723157 2024-03-04T17:50:35.189 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.188+0000 7fe960f568c0 1 < completed write operation key=7Z7BPZ8XGBHZ tag=2CYUJRYX0HA0 type=write size=658720 2024-03-04T17:50:35.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.196+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~QH907DNMIZT2.0 size=1048576 2024-03-04T17:50:35.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.196+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~QH907DNMIZT2.1 size=67329 2024-03-04T17:50:35.199 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.196+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=QH907DNMIZT2 tag=U9VIFDPBIN86 type=write size=2164481 2024-03-04T17:50:35.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.200+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~BVB1UBSU32E7.0 size=1048576 2024-03-04T17:50:35.206 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.204+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~BVB1UBSU32E7.1 size=478196 2024-03-04T17:50:35.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.212+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=BVB1UBSU32E7 tag=ND2T0KPZT9R2 type=write size=2575348 2024-03-04T17:50:35.219 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.216+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=0VEJSRF5Q7KP type=del size=3668352 2024-03-04T17:50:35.220 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.220+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=8IU9AU7MG3T7 type=del 2024-03-04T17:50:35.226 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.224+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~SXSRUBFD2IG6.0 size=1048576 2024-03-04T17:50:35.229 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.228+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~SXSRUBFD2IG6.1 size=1048576 2024-03-04T17:50:35.231 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.232+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~SXSRUBFD2IG6.2 size=569098 2024-03-04T17:50:35.232 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.232+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=SXSRUBFD2IG6 tag=G1Z1DKH5NCFW type=write size=3714826 2024-03-04T17:50:35.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.236+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=7I6RAHA305E2 type=del 2024-03-04T17:50:35.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.236+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=C743VT1Y6RPW type=del size=3096473 2024-03-04T17:50:35.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.240+0000 7fe960f568c0 1 < completed delete operation key=KYSLL05AUHAC tag=RP71C39H2J5L type=del 2024-03-04T17:50:35.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.244+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=OX20A4RF53LX type=del size=1441047 2024-03-04T17:50:35.253 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.252+0000 7fe960f568c0 1 < suggested update operation key=78N8HHC4O4CG tag=F08WFZC3V1HB type=del size=776377 2024-03-04T17:50:35.260 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.260+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~S9Q342PF8JFG.0 size=1048576 2024-03-04T17:50:35.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.260+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~S9Q342PF8JFG.1 size=745883 2024-03-04T17:50:35.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.260+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=S9Q342PF8JFG tag=REXZLB34AJRQ type=write size=2843035 2024-03-04T17:50:35.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.264+0000 7fe960f568c0 1 < canceled operation key=6X4ZV3BFZNDL tag=39APZCJG05GJ type=del size=274308 2024-03-04T17:50:35.269 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.268+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=VPMH22PLLOJD type=del size=2311781 2024-03-04T17:50:35.273 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.272+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=XE53DXCBZD4F type=del 2024-03-04T17:50:35.279 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.280+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~G711198RPXMZ.0 size=1048576 2024-03-04T17:50:35.282 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.280+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~G711198RPXMZ.1 size=1048576 2024-03-04T17:50:35.285 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.284+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~G711198RPXMZ.2 size=204746 2024-03-04T17:50:35.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.284+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=G711198RPXMZ tag=7WEH2TJX9586 type=write size=3350474 2024-03-04T17:50:35.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.288+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~8F1MAHTX7RI5.0 size=1048576 2024-03-04T17:50:35.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.292+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~8F1MAHTX7RI5.1 size=719951 2024-03-04T17:50:35.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.296+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=8F1MAHTX7RI5 tag=K5H5NZ8RU285 type=write size=2817103 2024-03-04T17:50:35.302 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.300+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=RZ6MGXHWWHOV type=del size=2653384 2024-03-04T17:50:35.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.304+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=2U09OZY6BELI type=del 2024-03-04T17:50:35.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.308+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=WU9YWT89VN1Z type=del size=4091851 2024-03-04T17:50:35.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.312+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=X4QSYPGC5YGM type=del 2024-03-04T17:50:35.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.316+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=ZCACDURYG8DA type=write size=953468 2024-03-04T17:50:35.320 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.320+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~0QCR48Q91ZG7.0 size=498692 2024-03-04T17:50:35.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.324+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=0QCR48Q91ZG7 tag=OGX63JUBHSWU type=write size=1547268 2024-03-04T17:50:35.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.328+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=QG5MI184G1FV type=del size=744883 2024-03-04T17:50:35.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.332+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=GUVQ5JL92KLT type=del 2024-03-04T17:50:35.336 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.336+0000 7fe960f568c0 1 > prepared operation key=KYSLL05AUHAC tag=O3BW0SXRWN7Z type=write size=702502 2024-03-04T17:50:35.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.340+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~XDHFJ951T98X.0 size=1048576 2024-03-04T17:50:35.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.340+0000 7fe960f568c0 1 < canceled part key=_multipart_NF4OIJWJ83DE.2~XDHFJ951T98X.1 size=157244 2024-03-04T17:50:35.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.344+0000 7fe960f568c0 1 < canceled multipart upload key=NF4OIJWJ83DE upload=XDHFJ951T98X tag=T35TBUW3NX45 type=write size=2254396 2024-03-04T17:50:35.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.348+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=1I58AP91TIL0 type=del size=2274224 2024-03-04T17:50:35.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.352+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~2OQSW0XVVJW6.0 size=1048576 2024-03-04T17:50:35.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.356+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~2OQSW0XVVJW6.1 size=1048576 2024-03-04T17:50:35.360 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.360+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~2OQSW0XVVJW6.2 size=11352 2024-03-04T17:50:35.366 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.364+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=2OQSW0XVVJW6 tag=VG7LD9P1RERY type=write size=3157080 2024-03-04T17:50:35.373 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.372+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~EQRFD7NGAK14.0 size=783415 2024-03-04T17:50:35.373 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.372+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=EQRFD7NGAK14 tag=DFHWSHQ858ZL type=write size=1831991 2024-03-04T17:50:35.377 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.376+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=C3SN9289WNPF type=del 2024-03-04T17:50:35.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.384+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=XTUUXMD0J2BP type=del size=2995307 2024-03-04T17:50:35.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.384+0000 7fe960f568c0 1 < canceled part key=_multipart_7Z7BPZ8XGBHZ.2~QH907DNMIZT2.0 size=1048576 2024-03-04T17:50:35.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.384+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~QH907DNMIZT2.1 size=67329 2024-03-04T17:50:35.392 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.392+0000 7fe960f568c0 1 < canceled multipart upload key=7Z7BPZ8XGBHZ upload=QH907DNMIZT2 tag=U9VIFDPBIN86 type=write size=2164481 2024-03-04T17:50:35.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.396+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~I4GU8FXUTU33.0 size=1048576 2024-03-04T17:50:35.405 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.404+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~I4GU8FXUTU33.1 size=446421 2024-03-04T17:50:35.405 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.404+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=I4GU8FXUTU33 tag=XTSCSL0F6BNB type=write size=2543573 2024-03-04T17:50:35.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.408+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=0VEJSRF5Q7KP type=del 2024-03-04T17:50:35.413 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.412+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=SPCX0G9KSHX2 type=write size=532796 2024-03-04T17:50:35.416 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.416+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~SXSRUBFD2IG6.0 size=1048576 2024-03-04T17:50:35.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.420+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~SXSRUBFD2IG6.1 size=1048576 2024-03-04T17:50:35.423 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.420+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~SXSRUBFD2IG6.2 size=569098 2024-03-04T17:50:35.428 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.428+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=SXSRUBFD2IG6 tag=G1Z1DKH5NCFW type=write size=3714826 2024-03-04T17:50:35.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.432+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=XGXFWQPH8J87 type=del size=232974 2024-03-04T17:50:35.436 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.436+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=C743VT1Y6RPW type=del 2024-03-04T17:50:35.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.440+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~LXLAYLVQ23W3.0 size=1048576 2024-03-04T17:50:35.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.444+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~LXLAYLVQ23W3.1 size=1048576 2024-03-04T17:50:35.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.448+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~LXLAYLVQ23W3.2 size=462324 2024-03-04T17:50:35.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.448+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=LXLAYLVQ23W3 tag=800HBVJUKCXB type=write size=3608052 2024-03-04T17:50:35.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.452+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=OX20A4RF53LX type=del 2024-03-04T17:50:35.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.460+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~BMPTHHPRRGX4.0 size=1048576 2024-03-04T17:50:35.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.464+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~BMPTHHPRRGX4.1 size=1048576 2024-03-04T17:50:35.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.464+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~BMPTHHPRRGX4.2 size=260703 2024-03-04T17:50:35.467 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.464+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=BMPTHHPRRGX4 tag=3Q0SECNT4DMV type=write size=3406431 2024-03-04T17:50:35.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.468+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~S9Q342PF8JFG.0 size=1048576 2024-03-04T17:50:35.474 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.472+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~S9Q342PF8JFG.1 size=745883 2024-03-04T17:50:35.480 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.480+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=S9Q342PF8JFG tag=REXZLB34AJRQ type=write size=2843035 2024-03-04T17:50:35.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.484+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=IUKZCX487SJE type=write size=753432 2024-03-04T17:50:35.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.488+0000 7fe960f568c0 1 < suggested update operation key=QFF67JIW8Y8E tag=VPMH22PLLOJD type=del size=2311781 2024-03-04T17:50:35.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.492+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=IO5K54FGS2AI type=del size=3373290 2024-03-04T17:50:35.499 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.496+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~G711198RPXMZ.0 size=1048576 2024-03-04T17:50:35.502 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.500+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~G711198RPXMZ.1 size=1048576 2024-03-04T17:50:35.504 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.504+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~G711198RPXMZ.2 size=204746 2024-03-04T17:50:35.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.508+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=G711198RPXMZ tag=7WEH2TJX9586 type=write size=3350474 2024-03-04T17:50:35.514 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.512+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=S1ODZC3SY6JJ type=write size=442327 2024-03-04T17:50:35.517 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.516+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=RZ6MGXHWWHOV type=del 2024-03-04T17:50:35.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.520+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=1D9GZ0E2GNA0 type=del size=434670 2024-03-04T17:50:35.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.524+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=WU9YWT89VN1Z type=del 2024-03-04T17:50:35.529 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.528+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=GM1G6HLTX2OA type=del size=1309586 2024-03-04T17:50:35.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.532+0000 7fe960f568c0 1 < completed write operation key=3X4719GHK3BI tag=ZCACDURYG8DA type=write size=953468 2024-03-04T17:50:35.536 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.536+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=GEFEP9OZ16GF type=del size=499386 2024-03-04T17:50:35.544 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.544+0000 7fe960f568c0 1 < suggested update operation key=UEVMIPQ9FR81 tag=QG5MI184G1FV type=del size=744883 2024-03-04T17:50:35.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.548+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=S8HSIEQTF09B type=del size=2060863 2024-03-04T17:50:35.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.552+0000 7fe960f568c0 1 < completed write operation key=KYSLL05AUHAC tag=O3BW0SXRWN7Z type=write size=702502 2024-03-04T17:50:35.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.552+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=QQSLS51YAM6A type=del size=2430098 2024-03-04T17:50:35.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.556+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=1I58AP91TIL0 type=del 2024-03-04T17:50:35.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.564+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~NFD3XDR5DQHB.0 size=1048576 2024-03-04T17:50:35.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.568+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~NFD3XDR5DQHB.1 size=1048576 2024-03-04T17:50:35.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.572+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~NFD3XDR5DQHB.2 size=677450 2024-03-04T17:50:35.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.572+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=NFD3XDR5DQHB tag=ZRJL02GC28DL type=write size=3823178 2024-03-04T17:50:35.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.572+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~EQRFD7NGAK14.0 size=783415 2024-03-04T17:50:35.581 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.580+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=EQRFD7NGAK14 tag=DFHWSHQ858ZL type=write size=1831991 2024-03-04T17:50:35.588 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.588+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~PMH3Y71WPQSV.0 size=1048576 2024-03-04T17:50:35.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.588+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~PMH3Y71WPQSV.1 size=1048576 2024-03-04T17:50:35.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.592+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~PMH3Y71WPQSV.2 size=42960 2024-03-04T17:50:35.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.592+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=PMH3Y71WPQSV tag=02P22A3VLQ85 type=write size=3188688 2024-03-04T17:50:35.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.596+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=XTUUXMD0J2BP type=del 2024-03-04T17:50:35.605 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.604+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~DRMMBLN0PJLX.0 size=1048576 2024-03-04T17:50:35.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.608+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~DRMMBLN0PJLX.1 size=1048576 2024-03-04T17:50:35.611 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.608+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~DRMMBLN0PJLX.2 size=431986 2024-03-04T17:50:35.611 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.608+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=DRMMBLN0PJLX tag=I7VM6VC4VNE3 type=write size=3577714 2024-03-04T17:50:35.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.616+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~I4GU8FXUTU33.0 size=1048576 2024-03-04T17:50:35.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.616+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~I4GU8FXUTU33.1 size=446421 2024-03-04T17:50:35.628 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.624+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=I4GU8FXUTU33 tag=XTSCSL0F6BNB type=write size=2543573 2024-03-04T17:50:35.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.632+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~Q9H8CCYCN90B.0 size=1048576 2024-03-04T17:50:35.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.632+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~Q9H8CCYCN90B.1 size=684308 2024-03-04T17:50:35.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.632+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=Q9H8CCYCN90B tag=LOWQHJF4EUXR type=write size=2781460 2024-03-04T17:50:35.639 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.636+0000 7fe960f568c0 1 < completed write operation key=FSWBX523X9V5 tag=SPCX0G9KSHX2 type=write size=532796 2024-03-04T17:50:35.643 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.640+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=25IIO58M8OP6 type=write size=89398 2024-03-04T17:50:35.647 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.648+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=XGXFWQPH8J87 type=del 2024-03-04T17:50:35.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.652+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~6OQ03FX3NHFK.0 size=511443 2024-03-04T17:50:35.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.652+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=6OQ03FX3NHFK tag=9ZHNDPIL3EWJ type=write size=1560019 2024-03-04T17:50:35.658 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.656+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~LXLAYLVQ23W3.0 size=1048576 2024-03-04T17:50:35.661 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.660+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~LXLAYLVQ23W3.1 size=1048576 2024-03-04T17:50:35.664 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.664+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~LXLAYLVQ23W3.2 size=462324 2024-03-04T17:50:35.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.668+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=LXLAYLVQ23W3 tag=800HBVJUKCXB type=write size=3608052 2024-03-04T17:50:35.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.676+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=XBN0NX47QFPE type=del size=3529420 2024-03-04T17:50:35.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.680+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~BMPTHHPRRGX4.0 size=1048576 2024-03-04T17:50:35.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.684+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~BMPTHHPRRGX4.1 size=1048576 2024-03-04T17:50:35.686 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.684+0000 7fe960f568c0 1 < completed part key=_multipart_NF4OIJWJ83DE.2~BMPTHHPRRGX4.2 size=260703 2024-03-04T17:50:35.692 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.692+0000 7fe960f568c0 1 < completed multipart upload key=NF4OIJWJ83DE upload=BMPTHHPRRGX4 tag=3Q0SECNT4DMV type=write size=3406431 2024-03-04T17:50:35.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.696+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=KTTIZ3X0JAAZ type=del size=2557285 2024-03-04T17:50:35.703 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.704+0000 7fe960f568c0 1 < suggested remove operation key=78N8HHC4O4CG tag=IUKZCX487SJE type=write 2024-03-04T17:50:35.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.704+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=9HHU3CURYE9E type=del size=3948554 2024-03-04T17:50:35.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.712+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=IO5K54FGS2AI type=del 2024-03-04T17:50:35.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.716+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~89K64X026UVG.0 size=795924 2024-03-04T17:50:35.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.716+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=89K64X026UVG tag=UX2L58GANKU2 type=write size=1844500 2024-03-04T17:50:35.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.720+0000 7fe960f568c0 1 < completed write operation key=RII14ZGX2EL1 tag=S1ODZC3SY6JJ type=write size=442327 2024-03-04T17:50:35.726 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.724+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=EME0YPLFMTB8 type=del size=1082664 2024-03-04T17:50:35.730 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.728+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=1D9GZ0E2GNA0 type=del 2024-03-04T17:50:35.737 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.736+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~7OUZKEGGOZI2.0 size=1048576 2024-03-04T17:50:35.740 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.740+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~7OUZKEGGOZI2.1 size=1048576 2024-03-04T17:50:35.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.744+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~7OUZKEGGOZI2.2 size=471588 2024-03-04T17:50:35.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.744+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=7OUZKEGGOZI2 tag=XE2S0R2GD3L8 type=write size=3617316 2024-03-04T17:50:35.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.748+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=GM1G6HLTX2OA type=del 2024-03-04T17:50:35.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.752+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=DTXAIFEC7G7S type=del size=3670885 2024-03-04T17:50:35.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.756+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=GEFEP9OZ16GF type=del 2024-03-04T17:50:35.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.764+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=EL4L88BNXYHY type=del size=2457504 2024-03-04T17:50:35.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.768+0000 7fe960f568c0 1 < suggested remove operation key=QFF67JIW8Y8E tag=S8HSIEQTF09B type=del 2024-03-04T17:50:35.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.776+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~BW8VI97FE7JY.0 size=398449 2024-03-04T17:50:35.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.776+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=BW8VI97FE7JY tag=QZF1QSO7ZKGW type=write size=1447025 2024-03-04T17:50:35.781 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.780+0000 7fe960f568c0 1 < canceled operation key=JT02WL9JYTNM tag=QQSLS51YAM6A type=del size=2430098 2024-03-04T17:50:35.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.784+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=HT6EKWPMVQPZ type=write size=476195 2024-03-04T17:50:35.788 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.788+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~NFD3XDR5DQHB.0 size=1048576 2024-03-04T17:50:35.791 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.788+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~NFD3XDR5DQHB.1 size=1048576 2024-03-04T17:50:35.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.792+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~NFD3XDR5DQHB.2 size=677450 2024-03-04T17:50:35.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.800+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=NFD3XDR5DQHB tag=ZRJL02GC28DL type=write size=3823178 2024-03-04T17:50:35.803 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.804+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=H939V2KV7MPO type=del size=3110303 2024-03-04T17:50:35.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.808+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~PMH3Y71WPQSV.0 size=1048576 2024-03-04T17:50:35.810 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.808+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~PMH3Y71WPQSV.1 size=1048576 2024-03-04T17:50:35.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.812+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~PMH3Y71WPQSV.2 size=42960 2024-03-04T17:50:35.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.820+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=PMH3Y71WPQSV tag=02P22A3VLQ85 type=write size=3188688 2024-03-04T17:50:35.823 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.820+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=X9RP05HOX4X7 type=del size=933719 2024-03-04T17:50:35.827 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.824+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~DRMMBLN0PJLX.0 size=1048576 2024-03-04T17:50:35.830 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.828+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~DRMMBLN0PJLX.1 size=1048576 2024-03-04T17:50:35.833 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.832+0000 7fe960f568c0 1 < completed part key=_multipart_2MKQA5JYWU8L.2~DRMMBLN0PJLX.2 size=431986 2024-03-04T17:50:35.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.836+0000 7fe960f568c0 1 < completed multipart upload key=2MKQA5JYWU8L upload=DRMMBLN0PJLX tag=I7VM6VC4VNE3 type=write size=3577714 2024-03-04T17:50:35.843 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.840+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=X28V2DVTZESS type=del size=2766229 2024-03-04T17:50:35.847 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.844+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~Q9H8CCYCN90B.0 size=1048576 2024-03-04T17:50:35.847 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.844+0000 7fe960f568c0 1 < canceled part key=_multipart_8PZD01OOX3AF.2~Q9H8CCYCN90B.1 size=684308 2024-03-04T17:50:35.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.852+0000 7fe960f568c0 1 < canceled multipart upload key=8PZD01OOX3AF upload=Q9H8CCYCN90B tag=LOWQHJF4EUXR type=write size=2781460 2024-03-04T17:50:35.861 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.860+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~NUPQIF63BJO8.0 size=707928 2024-03-04T17:50:35.861 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.860+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=NUPQIF63BJO8 tag=41K5XK2VNQ5X type=write size=1756504 2024-03-04T17:50:35.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.864+0000 7fe960f568c0 1 < completed write operation key=8PZD01OOX3AF tag=25IIO58M8OP6 type=write size=89398 2024-03-04T17:50:35.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.868+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=1F768VBCIJMH type=del size=3402246 2024-03-04T17:50:35.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.872+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~6OQ03FX3NHFK.0 size=511443 2024-03-04T17:50:35.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.880+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=6OQ03FX3NHFK tag=9ZHNDPIL3EWJ type=write size=1560019 2024-03-04T17:50:35.886 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.884+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~9L89ZIQB5KVO.0 size=1048576 2024-03-04T17:50:35.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.888+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~9L89ZIQB5KVO.1 size=510106 2024-03-04T17:50:35.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.888+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=9L89ZIQB5KVO tag=MM3AEQUXZ3F4 type=write size=2607258 2024-03-04T17:50:35.897 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.896+0000 7fe960f568c0 1 < suggested update operation key=7Z7BPZ8XGBHZ tag=XBN0NX47QFPE type=del size=3529420 2024-03-04T17:50:35.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.900+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=WTMOL386E4LD type=del size=1654570 2024-03-04T17:50:35.904 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.904+0000 7fe960f568c0 -1 < failed to complete operation key=QFF67JIW8Y8E tag=KTTIZ3X0JAAZ type=del size=2557285: (22) Invalid argument 2024-03-04T17:50:35.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.908+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=OBJ7ZDE2DT1V type=del size=1862941 2024-03-04T17:50:35.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.912+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=9HHU3CURYE9E type=del 2024-03-04T17:50:35.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.916+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~4GEPAD7WIYWJ.0 size=1048576 2024-03-04T17:50:35.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.920+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~4GEPAD7WIYWJ.1 size=1048576 2024-03-04T17:50:35.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.924+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~4GEPAD7WIYWJ.2 size=1001404 2024-03-04T17:50:35.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.924+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=4GEPAD7WIYWJ tag=ABX568ANXYMJ type=write size=4147132 2024-03-04T17:50:35.927 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.924+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~89K64X026UVG.0 size=795924 2024-03-04T17:50:35.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.932+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=89K64X026UVG tag=UX2L58GANKU2 type=write size=1844500 2024-03-04T17:50:35.936 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.936+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=VE2KVDZOZGPC type=del size=2358533 2024-03-04T17:50:35.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.940+0000 7fe960f568c0 1 < completed delete operation key=NF4OIJWJ83DE tag=EME0YPLFMTB8 type=del 2024-03-04T17:50:35.947 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.944+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~YJG5NYQ02R4Y.0 size=1048576 2024-03-04T17:50:35.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.948+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~YJG5NYQ02R4Y.1 size=1048576 2024-03-04T17:50:35.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.952+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~YJG5NYQ02R4Y.2 size=106900 2024-03-04T17:50:35.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.952+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=YJG5NYQ02R4Y tag=EMMKD0B2WK9O type=write size=3252628 2024-03-04T17:50:35.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.956+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~7OUZKEGGOZI2.0 size=1048576 2024-03-04T17:50:35.959 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.956+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~7OUZKEGGOZI2.1 size=1048576 2024-03-04T17:50:35.962 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.960+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~7OUZKEGGOZI2.2 size=471588 2024-03-04T17:50:35.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.968+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=7OUZKEGGOZI2 tag=XE2S0R2GD3L8 type=write size=3617316 2024-03-04T17:50:35.972 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.972+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=35R7OBYM9MCR type=write size=1005264 2024-03-04T17:50:35.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.976+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=DTXAIFEC7G7S type=del 2024-03-04T17:50:35.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.980+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=4W28XAK6NXIN type=del size=1850332 2024-03-04T17:50:35.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.984+0000 7fe960f568c0 1 < completed delete operation key=U2G6YEVD5506 tag=EL4L88BNXYHY type=del 2024-03-04T17:50:35.988 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.988+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=94O6DAD46LBP type=del size=2926520 2024-03-04T17:50:35.992 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.992+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~BW8VI97FE7JY.0 size=398449 2024-03-04T17:50:35.998 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:35.996+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=BW8VI97FE7JY tag=QZF1QSO7ZKGW type=write size=1447025 2024-03-04T17:50:36.002 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.000+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=EY1MWL7SWKF0 type=del size=1990048 2024-03-04T17:50:36.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.004+0000 7fe960f568c0 1 < completed write operation key=6X4ZV3BFZNDL tag=HT6EKWPMVQPZ type=write size=476195 2024-03-04T17:50:36.013 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.012+0000 7fe960f568c0 1 > prepared part key=_multipart_U2G6YEVD5506.2~A1KA954KJX3R.0 size=877090 2024-03-04T17:50:36.013 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.012+0000 7fe960f568c0 1 > prepared multipart upload key=U2G6YEVD5506 upload=A1KA954KJX3R tag=4F5XST7P0BJJ type=write size=1925666 2024-03-04T17:50:36.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.020+0000 7fe960f568c0 1 < suggested remove operation key=FSWBX523X9V5 tag=H939V2KV7MPO type=del 2024-03-04T17:50:36.027 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.028+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~F1I2F2EDPMIP.0 size=791019 2024-03-04T17:50:36.027 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.028+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=F1I2F2EDPMIP tag=1SHM0H80TFN8 type=write size=1839595 2024-03-04T17:50:36.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.032+0000 7fe960f568c0 1 < canceled operation key=JP3FRB6MD7B4 tag=X9RP05HOX4X7 type=del size=933719 2024-03-04T17:50:36.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.036+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~54S8MSFVFZYR.0 size=1048576 2024-03-04T17:50:36.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.040+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~54S8MSFVFZYR.1 size=1048576 2024-03-04T17:50:36.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.044+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~54S8MSFVFZYR.2 size=373503 2024-03-04T17:50:36.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.044+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=54S8MSFVFZYR tag=OS0KJMPYY75G type=write size=3519231 2024-03-04T17:50:36.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.048+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=X28V2DVTZESS type=del 2024-03-04T17:50:36.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.056+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~Q16EBAPVI837.0 size=1048576 2024-03-04T17:50:36.060 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.060+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~Q16EBAPVI837.1 size=1048576 2024-03-04T17:50:36.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.060+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~Q16EBAPVI837.2 size=428324 2024-03-04T17:50:36.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.060+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=Q16EBAPVI837 tag=4WX1B38HBT27 type=write size=3574052 2024-03-04T17:50:36.067 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.064+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~NUPQIF63BJO8.0 size=707928 2024-03-04T17:50:36.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.072+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=NUPQIF63BJO8 tag=41K5XK2VNQ5X type=write size=1756504 2024-03-04T17:50:36.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.076+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=5HMSBB8CMAZY type=del size=3688334 2024-03-04T17:50:36.080 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.080+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=1F768VBCIJMH type=del 2024-03-04T17:50:36.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.084+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~1RB51Q7XL7NP.0 size=1048576 2024-03-04T17:50:36.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.088+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~1RB51Q7XL7NP.1 size=131365 2024-03-04T17:50:36.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.088+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=1RB51Q7XL7NP tag=VFEJBASO2EUS type=write size=2228517 2024-03-04T17:50:36.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.092+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~9L89ZIQB5KVO.0 size=1048576 2024-03-04T17:50:36.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.092+0000 7fe960f568c0 1 < canceled part key=_multipart_GZ4RRIR5CC26.2~9L89ZIQB5KVO.1 size=510106 2024-03-04T17:50:36.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.100+0000 7fe960f568c0 1 < canceled multipart upload key=GZ4RRIR5CC26 upload=9L89ZIQB5KVO tag=MM3AEQUXZ3F4 type=write size=2607258 2024-03-04T17:50:36.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.108+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=4MU09GW407TQ type=write size=961049 2024-03-04T17:50:36.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.108+0000 7fe960f568c0 1 < completed delete operation key=QFF67JIW8Y8E tag=WTMOL386E4LD type=del 2024-03-04T17:50:36.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.112+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=00JOZL4KL9FJ type=del size=183033 2024-03-04T17:50:36.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.116+0000 7fe960f568c0 1 < completed delete operation key=U2G6YEVD5506 tag=OBJ7ZDE2DT1V type=del 2024-03-04T17:50:36.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.120+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=Z5Q33LDX1MWW type=del size=45672 2024-03-04T17:50:36.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.124+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~4GEPAD7WIYWJ.0 size=1048576 2024-03-04T17:50:36.128 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.128+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~4GEPAD7WIYWJ.1 size=1048576 2024-03-04T17:50:36.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.132+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~4GEPAD7WIYWJ.2 size=1001404 2024-03-04T17:50:36.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.136+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=4GEPAD7WIYWJ tag=ABX568ANXYMJ type=write size=4147132 2024-03-04T17:50:36.141 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.140+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=JEX1AQ5IU2B1 type=del size=2617717 2024-03-04T17:50:36.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.144+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=VE2KVDZOZGPC type=del 2024-03-04T17:50:36.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.152+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~CW1Y3HZHUC5C.0 size=1048576 2024-03-04T17:50:36.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.156+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~CW1Y3HZHUC5C.1 size=1048576 2024-03-04T17:50:36.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.156+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~CW1Y3HZHUC5C.2 size=1026923 2024-03-04T17:50:36.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.156+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=CW1Y3HZHUC5C tag=HG95TVV63KFW type=write size=4172651 2024-03-04T17:50:36.162 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.160+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~YJG5NYQ02R4Y.0 size=1048576 2024-03-04T17:50:36.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.164+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~YJG5NYQ02R4Y.1 size=1048576 2024-03-04T17:50:36.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.168+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~YJG5NYQ02R4Y.2 size=106900 2024-03-04T17:50:36.174 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.172+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=YJG5NYQ02R4Y tag=EMMKD0B2WK9O type=write size=3252628 2024-03-04T17:50:36.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.176+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=XGY21TNI7M0L type=write size=134147 2024-03-04T17:50:36.182 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.180+0000 7fe960f568c0 1 < completed write operation key=RII14ZGX2EL1 tag=35R7OBYM9MCR type=write size=1005264 2024-03-04T17:50:36.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.184+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=YZ4U54DS3IAY type=del size=1952133 2024-03-04T17:50:36.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.192+0000 7fe960f568c0 1 < suggested update operation key=RII14ZGX2EL1 tag=4W28XAK6NXIN type=del size=1850332 2024-03-04T17:50:36.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.196+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=ZJ9QEGP75H80 type=del size=2856646 2024-03-04T17:50:36.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.200+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=94O6DAD46LBP type=del 2024-03-04T17:50:36.206 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.204+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=UUKXBU2C85W4 type=write size=634849 2024-03-04T17:50:36.210 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.208+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=EY1MWL7SWKF0 type=del 2024-03-04T17:50:36.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.212+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=GTNZZS1KG0OJ type=del size=1783230 2024-03-04T17:50:36.218 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.216+0000 7fe960f568c0 1 < completed part key=_multipart_U2G6YEVD5506.2~A1KA954KJX3R.0 size=877090 2024-03-04T17:50:36.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.224+0000 7fe960f568c0 1 < completed multipart upload key=U2G6YEVD5506 upload=A1KA954KJX3R tag=4F5XST7P0BJJ type=write size=1925666 2024-03-04T17:50:36.227 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.224+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=X90LQBGQVU6E type=del size=834792 2024-03-04T17:50:36.231 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.228+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~F1I2F2EDPMIP.0 size=791019 2024-03-04T17:50:36.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.236+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=F1I2F2EDPMIP tag=1SHM0H80TFN8 type=write size=1839595 2024-03-04T17:50:36.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.240+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=5GS9BZC5PY0B type=del size=97206 2024-03-04T17:50:36.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.244+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~54S8MSFVFZYR.0 size=1048576 2024-03-04T17:50:36.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.248+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~54S8MSFVFZYR.1 size=1048576 2024-03-04T17:50:36.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.248+0000 7fe960f568c0 1 < canceled part key=_multipart_P28N6U79ZUV2.2~54S8MSFVFZYR.2 size=373503 2024-03-04T17:50:36.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.256+0000 7fe960f568c0 1 < canceled multipart upload key=P28N6U79ZUV2 upload=54S8MSFVFZYR tag=OS0KJMPYY75G type=write size=3519231 2024-03-04T17:50:36.259 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.260+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=ZXFSO2LDQAWR type=write size=71628 2024-03-04T17:50:36.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.260+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~Q16EBAPVI837.0 size=1048576 2024-03-04T17:50:36.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.264+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~Q16EBAPVI837.1 size=1048576 2024-03-04T17:50:36.268 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.268+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~Q16EBAPVI837.2 size=428324 2024-03-04T17:50:36.274 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.272+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=Q16EBAPVI837 tag=4WX1B38HBT27 type=write size=3574052 2024-03-04T17:50:36.278 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.276+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=CRJ2J8L3NQJM type=write size=629779 2024-03-04T17:50:36.282 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.280+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=5HMSBB8CMAZY type=del 2024-03-04T17:50:36.285 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.284+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=0FFMP3YV0Q13 type=del size=762947 2024-03-04T17:50:36.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.288+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~1RB51Q7XL7NP.0 size=1048576 2024-03-04T17:50:36.292 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.292+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~1RB51Q7XL7NP.1 size=131365 2024-03-04T17:50:36.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.296+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=1RB51Q7XL7NP tag=VFEJBASO2EUS type=write size=2228517 2024-03-04T17:50:36.306 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.304+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~3B55PD4IS3JA.0 size=1048576 2024-03-04T17:50:36.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.308+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~3B55PD4IS3JA.1 size=1048576 2024-03-04T17:50:36.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.312+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~3B55PD4IS3JA.2 size=334726 2024-03-04T17:50:36.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.312+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=3B55PD4IS3JA tag=UUQY33HXQ4IM type=write size=3480454 2024-03-04T17:50:36.320 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.320+0000 7fe960f568c0 1 < suggested remove operation key=2MKQA5JYWU8L tag=4MU09GW407TQ type=write 2024-03-04T17:50:36.327 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.328+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~E80RHDEB7TJE.0 size=1048576 2024-03-04T17:50:36.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.328+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~E80RHDEB7TJE.1 size=1048576 2024-03-04T17:50:36.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.332+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~E80RHDEB7TJE.2 size=478280 2024-03-04T17:50:36.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.332+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=E80RHDEB7TJE tag=7O13OAQECRPF type=write size=3624008 2024-03-04T17:50:36.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.336+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=00JOZL4KL9FJ type=del 2024-03-04T17:50:36.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.340+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=48KQT5UAOTDD type=del size=2309273 2024-03-04T17:50:36.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.344+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=Z5Q33LDX1MWW type=del 2024-03-04T17:50:36.350 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.348+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=C3HDF3FGR8CY type=del size=2840918 2024-03-04T17:50:36.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.352+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=JEX1AQ5IU2B1 type=del 2024-03-04T17:50:36.360 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.360+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~IXMX2XYSO9P4.0 size=1048576 2024-03-04T17:50:36.362 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.360+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~IXMX2XYSO9P4.1 size=1012462 2024-03-04T17:50:36.363 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.360+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=IXMX2XYSO9P4 tag=SGIODAWUGY5H type=write size=3109614 2024-03-04T17:50:36.366 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.364+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~CW1Y3HZHUC5C.0 size=1048576 2024-03-04T17:50:36.368 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.368+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~CW1Y3HZHUC5C.1 size=1048576 2024-03-04T17:50:36.371 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.372+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~CW1Y3HZHUC5C.2 size=1026923 2024-03-04T17:50:36.377 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.376+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=CW1Y3HZHUC5C tag=HG95TVV63KFW type=write size=4172651 2024-03-04T17:50:36.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.380+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=7DDUTPVCP90E type=del size=3207721 2024-03-04T17:50:36.384 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.384+0000 7fe960f568c0 1 < completed write operation key=GZ4RRIR5CC26 tag=XGY21TNI7M0L type=write size=134147 2024-03-04T17:50:36.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.388+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=UW29Q7T6IR4N type=del size=2132087 2024-03-04T17:50:36.392 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.392+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=YZ4U54DS3IAY type=del 2024-03-04T17:50:36.396 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.396+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=7SNTVXDPNVTC type=del size=864182 2024-03-04T17:50:36.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.400+0000 7fe960f568c0 1 < canceled operation key=JP3FRB6MD7B4 tag=ZJ9QEGP75H80 type=del size=2856646 2024-03-04T17:50:36.404 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.404+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=48ENVGIFVAQS type=write size=872823 2024-03-04T17:50:36.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.408+0000 7fe960f568c0 1 < completed write operation key=3X4719GHK3BI tag=UUKXBU2C85W4 type=write size=634849 2024-03-04T17:50:36.414 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.412+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~90G1AESHHHD4.0 size=128548 2024-03-04T17:50:36.414 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.412+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=90G1AESHHHD4 tag=X1WMI71AH5JS type=write size=1177124 2024-03-04T17:50:36.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.416+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=GTNZZS1KG0OJ type=del 2024-03-04T17:50:36.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.424+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~9YZ7MBP771P1.0 size=1048576 2024-03-04T17:50:36.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.428+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~9YZ7MBP771P1.1 size=70533 2024-03-04T17:50:36.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.428+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=9YZ7MBP771P1 tag=QY2801ONZH7G type=write size=2167685 2024-03-04T17:50:36.436 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.436+0000 7fe960f568c0 1 < suggested update operation key=R6ZDVOUE9NXJ tag=X90LQBGQVU6E type=del size=834792 2024-03-04T17:50:36.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.440+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=IC12J5HQ70SV type=del size=2351902 2024-03-04T17:50:36.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.444+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=5GS9BZC5PY0B type=del 2024-03-04T17:50:36.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.448+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=P9PNIQ9GO36D type=del size=3432143 2024-03-04T17:50:36.452 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.452+0000 7fe960f568c0 1 < canceled operation key=BFF9OD00TU54 tag=ZXFSO2LDQAWR type=write size=71628 2024-03-04T17:50:36.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.464+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~TZZS0ICU5284.0 size=1048576 2024-03-04T17:50:36.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.464+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~TZZS0ICU5284.1 size=920597 2024-03-04T17:50:36.467 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.464+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=TZZS0ICU5284 tag=VEYKO7V5VEO7 type=write size=3017749 2024-03-04T17:50:36.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.468+0000 7fe960f568c0 1 < completed write operation key=NF4OIJWJ83DE tag=CRJ2J8L3NQJM type=write size=629779 2024-03-04T17:50:36.474 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.472+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=INSWEJQP8UZR type=del size=1468 2024-03-04T17:50:36.478 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.476+0000 7fe960f568c0 1 < completed delete operation key=ASC735ZCBLCG tag=0FFMP3YV0Q13 type=del 2024-03-04T17:50:36.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.480+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=4L4CYCA72GFE type=del size=2323385 2024-03-04T17:50:36.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.484+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~3B55PD4IS3JA.0 size=1048576 2024-03-04T17:50:36.488 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.488+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~3B55PD4IS3JA.1 size=1048576 2024-03-04T17:50:36.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.492+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~3B55PD4IS3JA.2 size=334726 2024-03-04T17:50:36.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.496+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=3B55PD4IS3JA tag=UUQY33HXQ4IM type=write size=3480454 2024-03-04T17:50:36.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.504+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~U5CGINIQ0Y1B.0 size=1048576 2024-03-04T17:50:36.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.508+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~U5CGINIQ0Y1B.1 size=1048576 2024-03-04T17:50:36.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.508+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~U5CGINIQ0Y1B.2 size=56685 2024-03-04T17:50:36.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.508+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=U5CGINIQ0Y1B tag=LG0899YGBOUD type=write size=3202413 2024-03-04T17:50:36.515 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.516+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~E80RHDEB7TJE.0 size=1048576 2024-03-04T17:50:36.518 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.516+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~E80RHDEB7TJE.1 size=1048576 2024-03-04T17:50:36.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.520+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~E80RHDEB7TJE.2 size=478280 2024-03-04T17:50:36.527 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.528+0000 7fe960f568c0 1 < completed multipart upload key=XSQ3GE2UHMZG upload=E80RHDEB7TJE tag=7O13OAQECRPF type=write size=3624008 2024-03-04T17:50:36.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.532+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~R2C6TIWXOWMB.0 size=708275 2024-03-04T17:50:36.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.532+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=R2C6TIWXOWMB tag=YCHCCM9FLSW1 type=write size=1756851 2024-03-04T17:50:36.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.540+0000 7fe960f568c0 1 < suggested update operation key=2KZ4IFW54C5B tag=48KQT5UAOTDD type=del size=2309273 2024-03-04T17:50:36.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.544+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=ITEOG7HY4NGI type=del size=684334 2024-03-04T17:50:36.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.548+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=C3HDF3FGR8CY type=del 2024-03-04T17:50:36.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.556+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~501GGRMOA8FY.0 size=1048576 2024-03-04T17:50:36.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.556+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~501GGRMOA8FY.1 size=593031 2024-03-04T17:50:36.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.556+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=501GGRMOA8FY tag=L232BEHVRJCH type=write size=2690183 2024-03-04T17:50:36.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.560+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~IXMX2XYSO9P4.0 size=1048576 2024-03-04T17:50:36.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.564+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~IXMX2XYSO9P4.1 size=1012462 2024-03-04T17:50:36.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.572+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=IXMX2XYSO9P4 tag=SGIODAWUGY5H type=write size=3109614 2024-03-04T17:50:36.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.576+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~2K1ZH8YV09X2.0 size=152571 2024-03-04T17:50:36.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.576+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=2K1ZH8YV09X2 tag=CO8INN45UA1Z type=write size=1201147 2024-03-04T17:50:36.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.580+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=7DDUTPVCP90E type=del 2024-03-04T17:50:36.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.584+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=JDRWI57KT6U0 type=write size=192345 2024-03-04T17:50:36.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.592+0000 7fe960f568c0 1 < suggested remove operation key=U2G6YEVD5506 tag=UW29Q7T6IR4N type=del 2024-03-04T17:50:36.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.596+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=DEQ40HPR3C8G type=del size=2878857 2024-03-04T17:50:36.601 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.600+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=7SNTVXDPNVTC type=del 2024-03-04T17:50:36.605 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.604+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=RA6I3FX9HDKA type=del size=3086447 2024-03-04T17:50:36.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.608+0000 7fe960f568c0 1 < completed write operation key=BFF9OD00TU54 tag=48ENVGIFVAQS type=write size=872823 2024-03-04T17:50:36.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.616+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~F1KC6WR78Q1G.0 size=886345 2024-03-04T17:50:36.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.616+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=F1KC6WR78Q1G tag=66Y0TF6WSO8U type=write size=1934921 2024-03-04T17:50:36.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.620+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~90G1AESHHHD4.0 size=128548 2024-03-04T17:50:36.625 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.624+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=90G1AESHHHD4 tag=X1WMI71AH5JS type=write size=1177124 2024-03-04T17:50:36.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.632+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~Y4TAYB2JD4OU.0 size=1048576 2024-03-04T17:50:36.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.632+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~Y4TAYB2JD4OU.1 size=1048576 2024-03-04T17:50:36.638 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.636+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~Y4TAYB2JD4OU.2 size=781152 2024-03-04T17:50:36.638 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.636+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=Y4TAYB2JD4OU tag=6XOYKJPH73MN type=write size=3926880 2024-03-04T17:50:36.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.640+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~9YZ7MBP771P1.0 size=1048576 2024-03-04T17:50:36.644 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.644+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~9YZ7MBP771P1.1 size=70533 2024-03-04T17:50:36.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.648+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=9YZ7MBP771P1 tag=QY2801ONZH7G type=write size=2167685 2024-03-04T17:50:36.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.652+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=LCH7R0W2CCGB type=del size=4122803 2024-03-04T17:50:36.662 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.660+0000 7fe960f568c0 1 < suggested remove operation key=QFF67JIW8Y8E tag=IC12J5HQ70SV type=del 2024-03-04T17:50:36.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.668+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~TEO8Z18RVOHG.0 size=1048576 2024-03-04T17:50:36.672 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.672+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~TEO8Z18RVOHG.1 size=273585 2024-03-04T17:50:36.672 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.672+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=TEO8Z18RVOHG tag=Y4SWQI695B86 type=write size=2370737 2024-03-04T17:50:36.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.680+0000 7fe960f568c0 1 < suggested remove operation key=RII14ZGX2EL1 tag=P9PNIQ9GO36D type=del 2024-03-04T17:50:36.686 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.684+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~XJ3CJC0DFD3J.0 size=708983 2024-03-04T17:50:36.686 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.684+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=XJ3CJC0DFD3J tag=98HRE9O4MJ8E type=write size=1757559 2024-03-04T17:50:36.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.688+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~TZZS0ICU5284.0 size=1048576 2024-03-04T17:50:36.693 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.692+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~TZZS0ICU5284.1 size=920597 2024-03-04T17:50:36.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.696+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=TZZS0ICU5284 tag=VEYKO7V5VEO7 type=write size=3017749 2024-03-04T17:50:36.703 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.700+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=CT6ZK5LVMNAN type=del size=2766887 2024-03-04T17:50:36.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.704+0000 7fe960f568c0 -1 < failed to complete operation key=QFF67JIW8Y8E tag=INSWEJQP8UZR type=del size=1468: (22) Invalid argument 2024-03-04T17:50:36.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.712+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~DEYTWVOI7B8V.0 size=1048576 2024-03-04T17:50:36.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.716+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~DEYTWVOI7B8V.1 size=530376 2024-03-04T17:50:36.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.716+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=DEYTWVOI7B8V tag=XV0XNM4QER8H type=write size=2627528 2024-03-04T17:50:36.725 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.724+0000 7fe960f568c0 1 < suggested remove operation key=FLYDZHOIWO6L tag=4L4CYCA72GFE type=del 2024-03-04T17:50:36.729 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.728+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=L75IXQ69R01W type=del size=3384043 2024-03-04T17:50:36.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.732+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~U5CGINIQ0Y1B.0 size=1048576 2024-03-04T17:50:36.736 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.736+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~U5CGINIQ0Y1B.1 size=1048576 2024-03-04T17:50:36.740 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.740+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~U5CGINIQ0Y1B.2 size=56685 2024-03-04T17:50:36.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.744+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=U5CGINIQ0Y1B tag=LG0899YGBOUD type=write size=3202413 2024-03-04T17:50:36.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.752+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~KLR7OZONKOUJ.0 size=1048576 2024-03-04T17:50:36.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.756+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~KLR7OZONKOUJ.1 size=717685 2024-03-04T17:50:36.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.756+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=KLR7OZONKOUJ tag=EB0EVXZOEKE2 type=write size=2814837 2024-03-04T17:50:36.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.760+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~R2C6TIWXOWMB.0 size=708275 2024-03-04T17:50:36.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.768+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=R2C6TIWXOWMB tag=YCHCCM9FLSW1 type=write size=1756851 2024-03-04T17:50:36.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.772+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=19WUB9DRIOUF type=del size=3548439 2024-03-04T17:50:36.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.780+0000 7fe960f568c0 1 < suggested update operation key=RII14ZGX2EL1 tag=ITEOG7HY4NGI type=del size=684334 2024-03-04T17:50:36.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.784+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~H4LUK17CIA0I.0 size=1048576 2024-03-04T17:50:36.789 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.788+0000 7fe960f568c0 1 > prepared part key=_multipart_GZ4RRIR5CC26.2~H4LUK17CIA0I.1 size=253781 2024-03-04T17:50:36.789 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.788+0000 7fe960f568c0 1 > prepared multipart upload key=GZ4RRIR5CC26 upload=H4LUK17CIA0I tag=64PSIDKOVM7J type=write size=2350933 2024-03-04T17:50:36.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.788+0000 7fe960f568c0 1 < canceled part key=_multipart_WNXVI6K5MU3R.2~501GGRMOA8FY.0 size=1048576 2024-03-04T17:50:36.793 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.792+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~501GGRMOA8FY.1 size=593031 2024-03-04T17:50:36.799 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.800+0000 7fe960f568c0 1 < canceled multipart upload key=WNXVI6K5MU3R upload=501GGRMOA8FY tag=L232BEHVRJCH type=write size=2690183 2024-03-04T17:50:36.803 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.804+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=Q70RXOCAHKP2 type=write size=419689 2024-03-04T17:50:36.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.808+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~2K1ZH8YV09X2.0 size=152571 2024-03-04T17:50:36.818 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.816+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=2K1ZH8YV09X2 tag=CO8INN45UA1Z type=write size=1201147 2024-03-04T17:50:36.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.824+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~B9NP3IE5F0NM.0 size=1048576 2024-03-04T17:50:36.829 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.828+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~B9NP3IE5F0NM.1 size=1048576 2024-03-04T17:50:36.831 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.832+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~B9NP3IE5F0NM.2 size=557282 2024-03-04T17:50:36.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.832+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=B9NP3IE5F0NM tag=C2AUN5V75A4V type=write size=3703010 2024-03-04T17:50:36.835 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.836+0000 7fe960f568c0 1 < completed write operation key=GZ4RRIR5CC26 tag=JDRWI57KT6U0 type=write size=192345 2024-03-04T17:50:36.842 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.840+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~C5ME14RVQO4E.0 size=688170 2024-03-04T17:50:36.842 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.840+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=C5ME14RVQO4E tag=NESH9104HBMJ type=write size=1736746 2024-03-04T17:50:36.846 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.844+0000 7fe960f568c0 -1 < failed to complete operation key=RII14ZGX2EL1 tag=DEQ40HPR3C8G type=del size=2878857: (22) Invalid argument 2024-03-04T17:50:36.853 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.852+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~8RESKM6ANRDH.0 size=1048576 2024-03-04T17:50:36.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.856+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~8RESKM6ANRDH.1 size=1048576 2024-03-04T17:50:36.859 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.860+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~8RESKM6ANRDH.2 size=47109 2024-03-04T17:50:36.859 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.860+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=8RESKM6ANRDH tag=22NNFEUIPISG type=write size=3192837 2024-03-04T17:50:36.863 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.860+0000 7fe960f568c0 -1 < failed to complete operation key=RII14ZGX2EL1 tag=RA6I3FX9HDKA type=del size=3086447: (22) Invalid argument 2024-03-04T17:50:36.867 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.864+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=WPFJ048VNDOF type=write size=43242 2024-03-04T17:50:36.871 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.868+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~F1KC6WR78Q1G.0 size=886345 2024-03-04T17:50:36.876 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.876+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=F1KC6WR78Q1G tag=66Y0TF6WSO8U type=write size=1934921 2024-03-04T17:50:36.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.880+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~EXG9XKRFN42S.0 size=534683 2024-03-04T17:50:36.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.880+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=EXG9XKRFN42S tag=080YRULFTLUY type=write size=1583259 2024-03-04T17:50:36.886 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.884+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~Y4TAYB2JD4OU.0 size=1048576 2024-03-04T17:50:36.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.888+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~Y4TAYB2JD4OU.1 size=1048576 2024-03-04T17:50:36.892 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.892+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~Y4TAYB2JD4OU.2 size=781152 2024-03-04T17:50:36.898 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.896+0000 7fe960f568c0 1 < completed multipart upload key=ASC735ZCBLCG upload=Y4TAYB2JD4OU tag=6XOYKJPH73MN type=write size=3926880 2024-03-04T17:50:36.902 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.900+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=KEMAJUWOYGKQ type=del size=1336630 2024-03-04T17:50:36.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.904+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=LCH7R0W2CCGB type=del 2024-03-04T17:50:36.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.908+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=Z3TNRTNDTYKC type=del size=791087 2024-03-04T17:50:36.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.912+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~TEO8Z18RVOHG.0 size=1048576 2024-03-04T17:50:36.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.912+0000 7fe960f568c0 1 < canceled part key=_multipart_KYSLL05AUHAC.2~TEO8Z18RVOHG.1 size=273585 2024-03-04T17:50:36.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.920+0000 7fe960f568c0 1 < canceled multipart upload key=KYSLL05AUHAC upload=TEO8Z18RVOHG tag=Y4SWQI695B86 type=write size=2370737 2024-03-04T17:50:36.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.924+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=TJUC4FOIBP6S type=del size=3460551 2024-03-04T17:50:36.927 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.928+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~XJ3CJC0DFD3J.0 size=708983 2024-03-04T17:50:36.933 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.932+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=XJ3CJC0DFD3J tag=98HRE9O4MJ8E type=write size=1757559 2024-03-04T17:50:36.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.940+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~3RKLQILNSYDE.0 size=1048576 2024-03-04T17:50:36.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.940+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~3RKLQILNSYDE.1 size=642327 2024-03-04T17:50:36.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.940+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=3RKLQILNSYDE tag=Z278UTCBPJ8L type=write size=2739479 2024-03-04T17:50:36.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.944+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=CT6ZK5LVMNAN type=del 2024-03-04T17:50:36.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.948+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=PPMCPL2UMGVZ type=del size=1454486 2024-03-04T17:50:36.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.956+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~DEYTWVOI7B8V.0 size=1048576 2024-03-04T17:50:36.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.956+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~DEYTWVOI7B8V.1 size=530376 2024-03-04T17:50:36.964 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.964+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=DEYTWVOI7B8V tag=XV0XNM4QER8H type=write size=2627528 2024-03-04T17:50:36.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.968+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=KLPT4JJM1F3F type=del size=4083129 2024-03-04T17:50:36.972 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.972+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=L75IXQ69R01W type=del 2024-03-04T17:50:36.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.980+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~HNCO0ZZM5NWR.0 size=1048576 2024-03-04T17:50:36.983 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.980+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~HNCO0ZZM5NWR.1 size=1048576 2024-03-04T17:50:36.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.984+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~HNCO0ZZM5NWR.2 size=182012 2024-03-04T17:50:36.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.984+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=HNCO0ZZM5NWR tag=2H1402XM6TMA type=write size=3327740 2024-03-04T17:50:36.990 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.988+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~KLR7OZONKOUJ.0 size=1048576 2024-03-04T17:50:36.993 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:36.992+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~KLR7OZONKOUJ.1 size=717685 2024-03-04T17:50:37.000 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.000+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=KLR7OZONKOUJ tag=EB0EVXZOEKE2 type=write size=2814837 2024-03-04T17:50:37.005 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.004+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=D58VA5HCJ4R6 type=del size=1513058 2024-03-04T17:50:37.013 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.012+0000 7fe960f568c0 1 < suggested update operation key=768X2XKWJW1S tag=19WUB9DRIOUF type=del size=3548439 2024-03-04T17:50:37.017 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.016+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=SF5GTQ659SP8 type=del size=3459768 2024-03-04T17:50:37.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.020+0000 7fe960f568c0 1 < completed part key=_multipart_GZ4RRIR5CC26.2~H4LUK17CIA0I.0 size=1048576 2024-03-04T17:50:37.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.020+0000 7fe960f568c0 1 < canceled part key=_multipart_GZ4RRIR5CC26.2~H4LUK17CIA0I.1 size=253781 2024-03-04T17:50:37.027 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.028+0000 7fe960f568c0 1 < canceled multipart upload key=GZ4RRIR5CC26 upload=H4LUK17CIA0I tag=64PSIDKOVM7J type=write size=2350933 2024-03-04T17:50:37.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.032+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~01RCZ5GWEGQR.0 size=194066 2024-03-04T17:50:37.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.032+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=01RCZ5GWEGQR tag=2NT273S011KX type=write size=1242642 2024-03-04T17:50:37.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.040+0000 7fe960f568c0 1 < completed write operation key=RII14ZGX2EL1 tag=Q70RXOCAHKP2 type=write size=419689 2024-03-04T17:50:37.043 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.040+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=8FWLMUAWE032 type=del size=1893975 2024-03-04T17:50:37.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.048+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~B9NP3IE5F0NM.0 size=1048576 2024-03-04T17:50:37.051 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.048+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~B9NP3IE5F0NM.1 size=1048576 2024-03-04T17:50:37.054 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.052+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~B9NP3IE5F0NM.2 size=557282 2024-03-04T17:50:37.060 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.060+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=B9NP3IE5F0NM tag=C2AUN5V75A4V type=write size=3703010 2024-03-04T17:50:37.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.064+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=Z67GH4W43U0G type=del size=504107 2024-03-04T17:50:37.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.064+0000 7fe960f568c0 1 < canceled part key=_multipart_FLYDZHOIWO6L.2~C5ME14RVQO4E.0 size=688170 2024-03-04T17:50:37.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.068+0000 7fe960f568c0 1 < canceled multipart upload key=FLYDZHOIWO6L upload=C5ME14RVQO4E tag=NESH9104HBMJ type=write size=1736746 2024-03-04T17:50:37.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.072+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=PQN0QHGVXCOD type=del size=4139498 2024-03-04T17:50:37.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.076+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~8RESKM6ANRDH.0 size=1048576 2024-03-04T17:50:37.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.076+0000 7fe960f568c0 1 < canceled part key=_multipart_768X2XKWJW1S.2~8RESKM6ANRDH.1 size=1048576 2024-03-04T17:50:37.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.080+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~8RESKM6ANRDH.2 size=47109 2024-03-04T17:50:37.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.088+0000 7fe960f568c0 1 < canceled multipart upload key=768X2XKWJW1S upload=8RESKM6ANRDH tag=22NNFEUIPISG type=write size=3192837 2024-03-04T17:50:37.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.092+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=V5SBU5A8NF3O type=del size=1441334 2024-03-04T17:50:37.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.100+0000 7fe960f568c0 1 < suggested remove operation key=7Z7BPZ8XGBHZ tag=WPFJ048VNDOF type=write 2024-03-04T17:50:37.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.108+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~NVHT3K53J8JX.0 size=1048576 2024-03-04T17:50:37.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.108+0000 7fe960f568c0 1 > prepared part key=_multipart_2FUTMG0SSXBY.2~NVHT3K53J8JX.1 size=955661 2024-03-04T17:50:37.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.108+0000 7fe960f568c0 1 > prepared multipart upload key=2FUTMG0SSXBY upload=NVHT3K53J8JX tag=XC2XITRYS06I type=write size=3052813 2024-03-04T17:50:37.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.112+0000 7fe960f568c0 1 < canceled part key=_multipart_FSWBX523X9V5.2~EXG9XKRFN42S.0 size=534683 2024-03-04T17:50:37.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.116+0000 7fe960f568c0 1 < canceled multipart upload key=FSWBX523X9V5 upload=EXG9XKRFN42S tag=080YRULFTLUY type=write size=1583259 2024-03-04T17:50:37.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.124+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~NEUZLEUVMC0K.0 size=1048576 2024-03-04T17:50:37.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.124+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~NEUZLEUVMC0K.1 size=603316 2024-03-04T17:50:37.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.124+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=NEUZLEUVMC0K tag=DF9NGGPE1JX7 type=write size=2700468 2024-03-04T17:50:37.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.128+0000 7fe960f568c0 1 < completed delete operation key=QGLN27WH1N2O tag=KEMAJUWOYGKQ type=del 2024-03-04T17:50:37.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.136+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~ONIPVBTL0F0G.0 size=1048576 2024-03-04T17:50:37.141 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.140+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~ONIPVBTL0F0G.1 size=1048576 2024-03-04T17:50:37.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.144+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~ONIPVBTL0F0G.2 size=33848 2024-03-04T17:50:37.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.144+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=ONIPVBTL0F0G tag=63ZL9G6CW1M8 type=write size=3179576 2024-03-04T17:50:37.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.148+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=Z3TNRTNDTYKC type=del 2024-03-04T17:50:37.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.156+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~B5Y5KWJ42T1W.0 size=1048576 2024-03-04T17:50:37.159 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.160+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~B5Y5KWJ42T1W.1 size=813641 2024-03-04T17:50:37.159 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.160+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=B5Y5KWJ42T1W tag=KT5F580ISLLF type=write size=2910793 2024-03-04T17:50:37.163 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.160+0000 7fe960f568c0 -1 < failed to complete operation key=7Z7BPZ8XGBHZ tag=TJUC4FOIBP6S type=del size=3460551: (22) Invalid argument 2024-03-04T17:50:37.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.172+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~A37C7CIC69EU.0 size=1048576 2024-03-04T17:50:37.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.176+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~A37C7CIC69EU.1 size=420014 2024-03-04T17:50:37.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.176+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=A37C7CIC69EU tag=S54INWR812Z1 type=write size=2517166 2024-03-04T17:50:37.182 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.180+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~3RKLQILNSYDE.0 size=1048576 2024-03-04T17:50:37.185 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.184+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~3RKLQILNSYDE.1 size=642327 2024-03-04T17:50:37.191 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.188+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=3RKLQILNSYDE tag=Z278UTCBPJ8L type=write size=2739479 2024-03-04T17:50:37.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.192+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=Q4ORB3B7TZOW type=del size=1889346 2024-03-04T17:50:37.197 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.196+0000 7fe960f568c0 -1 < failed to complete operation key=7Z7BPZ8XGBHZ tag=PPMCPL2UMGVZ type=del size=1454486: (22) Invalid argument 2024-03-04T17:50:37.201 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.200+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=R9UH7A33Q7QV type=del size=3340275 2024-03-04T17:50:37.205 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.204+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=KLPT4JJM1F3F type=del 2024-03-04T17:50:37.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.208+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=6T39DIJN8VEH type=write size=436235 2024-03-04T17:50:37.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.212+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~HNCO0ZZM5NWR.0 size=1048576 2024-03-04T17:50:37.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.212+0000 7fe960f568c0 1 < canceled part key=_multipart_6X4ZV3BFZNDL.2~HNCO0ZZM5NWR.1 size=1048576 2024-03-04T17:50:37.215 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.212+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~HNCO0ZZM5NWR.2 size=182012 2024-03-04T17:50:37.221 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.220+0000 7fe960f568c0 1 < canceled multipart upload key=6X4ZV3BFZNDL upload=HNCO0ZZM5NWR tag=2H1402XM6TMA type=write size=3327740 2024-03-04T17:50:37.225 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.224+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=1RG0V822Y6CU type=del size=1966611 2024-03-04T17:50:37.228 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.228+0000 7fe960f568c0 1 < canceled operation key=JP3FRB6MD7B4 tag=D58VA5HCJ4R6 type=del size=1513058 2024-03-04T17:50:37.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.232+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~TGHB4OAV28P5.0 size=1048576 2024-03-04T17:50:37.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.236+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~TGHB4OAV28P5.1 size=887726 2024-03-04T17:50:37.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.236+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=TGHB4OAV28P5 tag=S6VSIPTWBJTC type=write size=2984878 2024-03-04T17:50:37.241 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.240+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=SF5GTQ659SP8 type=del 2024-03-04T17:50:37.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.244+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=474AIX5N6BUM type=del size=1478497 2024-03-04T17:50:37.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.248+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~01RCZ5GWEGQR.0 size=194066 2024-03-04T17:50:37.254 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.252+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=01RCZ5GWEGQR tag=2NT273S011KX type=write size=1242642 2024-03-04T17:50:37.257 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.256+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=4XJ12R6RWD24 type=del size=639985 2024-03-04T17:50:37.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.260+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=8FWLMUAWE032 type=del 2024-03-04T17:50:37.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.264+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=ZQGDYYEWQPWG type=del size=3492339 2024-03-04T17:50:37.268 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.268+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=Z67GH4W43U0G type=del 2024-03-04T17:50:37.274 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.272+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~FZ0D2JATWLMK.0 size=1048576 2024-03-04T17:50:37.277 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.276+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~FZ0D2JATWLMK.1 size=1048576 2024-03-04T17:50:37.280 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.280+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~FZ0D2JATWLMK.2 size=433180 2024-03-04T17:50:37.280 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.280+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=FZ0D2JATWLMK tag=2IPZ7FYESCIF type=write size=3578908 2024-03-04T17:50:37.283 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.284+0000 7fe960f568c0 1 < canceled operation key=JP3FRB6MD7B4 tag=PQN0QHGVXCOD type=del size=4139498 2024-03-04T17:50:37.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.288+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~BS4VANMMTAFV.0 size=337175 2024-03-04T17:50:37.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.288+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=BS4VANMMTAFV tag=S12EL17HAS9Q type=write size=1385751 2024-03-04T17:50:37.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.292+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=V5SBU5A8NF3O type=del 2024-03-04T17:50:37.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.296+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=CW757VW8KINE type=del size=3089764 2024-03-04T17:50:37.301 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.300+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~NVHT3K53J8JX.0 size=1048576 2024-03-04T17:50:37.304 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.304+0000 7fe960f568c0 1 < completed part key=_multipart_2FUTMG0SSXBY.2~NVHT3K53J8JX.1 size=955661 2024-03-04T17:50:37.310 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.308+0000 7fe960f568c0 1 < completed multipart upload key=2FUTMG0SSXBY upload=NVHT3K53J8JX tag=XC2XITRYS06I type=write size=3052813 2024-03-04T17:50:37.314 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.312+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=7ESWN6GDDRVY type=write size=27204 2024-03-04T17:50:37.317 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.316+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~NEUZLEUVMC0K.0 size=1048576 2024-03-04T17:50:37.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.320+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~NEUZLEUVMC0K.1 size=603316 2024-03-04T17:50:37.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.324+0000 7fe960f568c0 1 < completed multipart upload key=R6ZDVOUE9NXJ upload=NEUZLEUVMC0K tag=DF9NGGPE1JX7 type=write size=2700468 2024-03-04T17:50:37.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.332+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~2EGZRN8D0COG.0 size=1048576 2024-03-04T17:50:37.334 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.332+0000 7fe960f568c0 1 > prepared part key=_multipart_MIZRUP4MEA0J.2~2EGZRN8D0COG.1 size=522317 2024-03-04T17:50:37.334 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.332+0000 7fe960f568c0 1 > prepared multipart upload key=MIZRUP4MEA0J upload=2EGZRN8D0COG tag=CFGECMADQO9Y type=write size=2619469 2024-03-04T17:50:37.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.336+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~ONIPVBTL0F0G.0 size=1048576 2024-03-04T17:50:37.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.340+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~ONIPVBTL0F0G.1 size=1048576 2024-03-04T17:50:37.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.344+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~ONIPVBTL0F0G.2 size=33848 2024-03-04T17:50:37.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.348+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=ONIPVBTL0F0G tag=63ZL9G6CW1M8 type=write size=3179576 2024-03-04T17:50:37.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.356+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~V9FDKWJITE9Z.0 size=684880 2024-03-04T17:50:37.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.356+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=V9FDKWJITE9Z tag=QN200GW5HACY type=write size=1733456 2024-03-04T17:50:37.360 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.360+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~B5Y5KWJ42T1W.0 size=1048576 2024-03-04T17:50:37.360 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.360+0000 7fe960f568c0 1 < canceled part key=_multipart_6X4ZV3BFZNDL.2~B5Y5KWJ42T1W.1 size=813641 2024-03-04T17:50:37.365 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.364+0000 7fe960f568c0 1 < canceled multipart upload key=6X4ZV3BFZNDL upload=B5Y5KWJ42T1W tag=KT5F580ISLLF type=write size=2910793 2024-03-04T17:50:37.369 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.368+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=LJYW5WK9AWLJ type=del size=3980758 2024-03-04T17:50:37.373 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.372+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~A37C7CIC69EU.0 size=1048576 2024-03-04T17:50:37.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.376+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~A37C7CIC69EU.1 size=420014 2024-03-04T17:50:37.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.380+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=A37C7CIC69EU tag=S54INWR812Z1 type=write size=2517166 2024-03-04T17:50:37.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.384+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=VQ40P92O1P4Q type=write size=839774 2024-03-04T17:50:37.390 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.388+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=Q4ORB3B7TZOW type=del 2024-03-04T17:50:37.394 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.392+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=MVWZENUUWUGT type=del size=733076 2024-03-04T17:50:37.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.396+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=R9UH7A33Q7QV type=del 2024-03-04T17:50:37.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.400+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=M09RVO9TBBRJ type=del size=3679927 2024-03-04T17:50:37.407 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.408+0000 7fe960f568c0 1 < canceled operation key=JT02WL9JYTNM tag=6T39DIJN8VEH type=write size=436235 2024-03-04T17:50:37.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.412+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=NUKNR1R0ELV6 type=write size=344940 2024-03-04T17:50:37.415 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.416+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=1RG0V822Y6CU type=del 2024-03-04T17:50:37.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.420+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=OTUUKT2NXUGS type=write size=424398 2024-03-04T17:50:37.423 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.424+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~TGHB4OAV28P5.0 size=1048576 2024-03-04T17:50:37.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.424+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~TGHB4OAV28P5.1 size=887726 2024-03-04T17:50:37.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.432+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=TGHB4OAV28P5 tag=S6VSIPTWBJTC type=write size=2984878 2024-03-04T17:50:37.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.440+0000 7fe960f568c0 1 > prepared part key=_multipart_QGLN27WH1N2O.2~NLAQ3W5RSAW3.0 size=28042 2024-03-04T17:50:37.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.440+0000 7fe960f568c0 1 > prepared multipart upload key=QGLN27WH1N2O upload=NLAQ3W5RSAW3 tag=5H0A8YUCWNKC type=write size=1076618 2024-03-04T17:50:37.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.444+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=474AIX5N6BUM type=del 2024-03-04T17:50:37.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.448+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~127QZ494Y1UO.0 size=56546 2024-03-04T17:50:37.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.448+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=127QZ494Y1UO tag=4F7E0C7BRP9L type=write size=1105122 2024-03-04T17:50:37.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.452+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=4XJ12R6RWD24 type=del 2024-03-04T17:50:37.460 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.460+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~FUKYIR4QY993.0 size=1048576 2024-03-04T17:50:37.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.464+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~FUKYIR4QY993.1 size=1034672 2024-03-04T17:50:37.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.464+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=FUKYIR4QY993 tag=U8MHOO923ORN type=write size=3131824 2024-03-04T17:50:37.468 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.468+0000 7fe960f568c0 1 < canceled operation key=U2G6YEVD5506 tag=ZQGDYYEWQPWG type=del size=3492339 2024-03-04T17:50:37.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.468+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=P0ZL55BI3U6M type=del size=3241316 2024-03-04T17:50:37.476 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.476+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~FZ0D2JATWLMK.0 size=1048576 2024-03-04T17:50:37.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.480+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~FZ0D2JATWLMK.1 size=1048576 2024-03-04T17:50:37.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.480+0000 7fe960f568c0 1 < completed part key=_multipart_WNXVI6K5MU3R.2~FZ0D2JATWLMK.2 size=433180 2024-03-04T17:50:37.488 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.488+0000 7fe960f568c0 1 < completed multipart upload key=WNXVI6K5MU3R upload=FZ0D2JATWLMK tag=2IPZ7FYESCIF type=write size=3578908 2024-03-04T17:50:37.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.492+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=A1F3HYON1OWR type=del size=3031569 2024-03-04T17:50:37.493 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.492+0000 7fe960f568c0 1 < canceled part key=_multipart_QFF67JIW8Y8E.2~BS4VANMMTAFV.0 size=337175 2024-03-04T17:50:37.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.496+0000 7fe960f568c0 1 < canceled multipart upload key=QFF67JIW8Y8E upload=BS4VANMMTAFV tag=S12EL17HAS9Q type=write size=1385751 2024-03-04T17:50:37.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.504+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~TC09HF0H6Q00.0 size=341757 2024-03-04T17:50:37.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.504+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=TC09HF0H6Q00 tag=BSB2G9QLR71F type=write size=1390333 2024-03-04T17:50:37.514 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.512+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=CW757VW8KINE type=del 2024-03-04T17:50:37.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.520+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~TH8WCM41ZKLS.0 size=1048576 2024-03-04T17:50:37.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.524+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~TH8WCM41ZKLS.1 size=1048576 2024-03-04T17:50:37.526 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.524+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~TH8WCM41ZKLS.2 size=1011279 2024-03-04T17:50:37.526 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.524+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=TH8WCM41ZKLS tag=WCE8NCPGG296 type=write size=4157007 2024-03-04T17:50:37.530 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.528+0000 7fe960f568c0 1 < canceled operation key=UKZ079SVZZKG tag=7ESWN6GDDRVY type=write size=27204 2024-03-04T17:50:37.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.532+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=5O80TU2QK8OB type=del size=4074658 2024-03-04T17:50:37.537 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.536+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~2EGZRN8D0COG.0 size=1048576 2024-03-04T17:50:37.540 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.540+0000 7fe960f568c0 1 < completed part key=_multipart_MIZRUP4MEA0J.2~2EGZRN8D0COG.1 size=522317 2024-03-04T17:50:37.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.544+0000 7fe960f568c0 1 < completed multipart upload key=MIZRUP4MEA0J upload=2EGZRN8D0COG tag=CFGECMADQO9Y type=write size=2619469 2024-03-04T17:50:37.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.548+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=IY2TJUHCPR8B type=write size=761263 2024-03-04T17:50:37.553 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.552+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~V9FDKWJITE9Z.0 size=684880 2024-03-04T17:50:37.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.560+0000 7fe960f568c0 1 < completed multipart upload key=ASC735ZCBLCG upload=V9FDKWJITE9Z tag=QN200GW5HACY type=write size=1733456 2024-03-04T17:50:37.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.564+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~Z67EMXINSX3X.0 size=367350 2024-03-04T17:50:37.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.564+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=Z67EMXINSX3X tag=T59C6K49F4UI type=write size=1415926 2024-03-04T17:50:37.570 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.568+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=LJYW5WK9AWLJ type=del 2024-03-04T17:50:37.574 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.572+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=16QF04JR23KM type=del size=1773754 2024-03-04T17:50:37.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.576+0000 7fe960f568c0 1 < completed write operation key=P28N6U79ZUV2 tag=VQ40P92O1P4Q type=write size=839774 2024-03-04T17:50:37.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.580+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=12B7MD774B9Q type=write size=124116 2024-03-04T17:50:37.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.584+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=MVWZENUUWUGT type=del 2024-03-04T17:50:37.590 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.588+0000 7fe960f568c0 1 > prepared operation key=QFF67JIW8Y8E tag=JA6WQVL5ZR9A type=write size=892643 2024-03-04T17:50:37.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.592+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=M09RVO9TBBRJ type=del 2024-03-04T17:50:37.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.596+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=L036AX4IDG7E type=del size=4183281 2024-03-04T17:50:37.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.600+0000 7fe960f568c0 1 < canceled operation key=JT02WL9JYTNM tag=NUKNR1R0ELV6 type=write size=344940 2024-03-04T17:50:37.610 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.608+0000 7fe960f568c0 1 > prepared part key=_multipart_JT02WL9JYTNM.2~5C7DV4N72DO9.0 size=511711 2024-03-04T17:50:37.610 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.608+0000 7fe960f568c0 1 > prepared multipart upload key=JT02WL9JYTNM upload=5C7DV4N72DO9 tag=20P00D6XYN08 type=write size=1560287 2024-03-04T17:50:37.613 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.612+0000 7fe960f568c0 1 < canceled operation key=768X2XKWJW1S tag=OTUUKT2NXUGS type=write size=424398 2024-03-04T17:50:37.617 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.616+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=FPE1N2JH3IU1 type=del size=769623 2024-03-04T17:50:37.621 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.620+0000 7fe960f568c0 1 < completed part key=_multipart_QGLN27WH1N2O.2~NLAQ3W5RSAW3.0 size=28042 2024-03-04T17:50:37.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.624+0000 7fe960f568c0 1 < completed multipart upload key=QGLN27WH1N2O upload=NLAQ3W5RSAW3 tag=5H0A8YUCWNKC type=write size=1076618 2024-03-04T17:50:37.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.632+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=G6KD0UR2N5Y7 type=del size=1391949 2024-03-04T17:50:37.635 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.636+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~127QZ494Y1UO.0 size=56546 2024-03-04T17:50:37.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.640+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=127QZ494Y1UO tag=4F7E0C7BRP9L type=write size=1105122 2024-03-04T17:50:37.644 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.644+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=OQZLKDFACFZL type=write size=717267 2024-03-04T17:50:37.648 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.648+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~FUKYIR4QY993.0 size=1048576 2024-03-04T17:50:37.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.648+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~FUKYIR4QY993.1 size=1034672 2024-03-04T17:50:37.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.656+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=FUKYIR4QY993 tag=U8MHOO923ORN type=write size=3131824 2024-03-04T17:50:37.661 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.660+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=C7Q8HZ19SDVZ type=del size=3132695 2024-03-04T17:50:37.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.664+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=P0ZL55BI3U6M type=del 2024-03-04T17:50:37.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.668+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=EHERKUXEMMF2 type=del size=2765041 2024-03-04T17:50:37.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.676+0000 7fe960f568c0 1 < suggested update operation key=7Z7BPZ8XGBHZ tag=A1F3HYON1OWR type=del size=3031569 2024-03-04T17:50:37.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.680+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=0AE90UKBD145 type=del size=3733054 2024-03-04T17:50:37.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.684+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~TC09HF0H6Q00.0 size=341757 2024-03-04T17:50:37.691 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.692+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=TC09HF0H6Q00 tag=BSB2G9QLR71F type=write size=1390333 2024-03-04T17:50:37.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.696+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~ET9LMT892Q6X.0 size=1048576 2024-03-04T17:50:37.702 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.700+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~ET9LMT892Q6X.1 size=680948 2024-03-04T17:50:37.702 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.700+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=ET9LMT892Q6X tag=5ZL0ZYHXY9UZ type=write size=2778100 2024-03-04T17:50:37.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.704+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~TH8WCM41ZKLS.0 size=1048576 2024-03-04T17:50:37.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.708+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~TH8WCM41ZKLS.1 size=1048576 2024-03-04T17:50:37.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.712+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~TH8WCM41ZKLS.2 size=1011279 2024-03-04T17:50:37.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.716+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=TH8WCM41ZKLS tag=WCE8NCPGG296 type=write size=4157007 2024-03-04T17:50:37.726 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.724+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~FZ5LWD78EOVT.0 size=1048576 2024-03-04T17:50:37.729 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.728+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~FZ5LWD78EOVT.1 size=115654 2024-03-04T17:50:37.729 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.728+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=FZ5LWD78EOVT tag=RMCK0MI6ULCP type=write size=2212806 2024-03-04T17:50:37.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.732+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=5O80TU2QK8OB type=del 2024-03-04T17:50:37.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.740+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~0KIMGN1H0U87.0 size=1048576 2024-03-04T17:50:37.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.744+0000 7fe960f568c0 1 > prepared part key=_multipart_R6ZDVOUE9NXJ.2~0KIMGN1H0U87.1 size=131094 2024-03-04T17:50:37.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.744+0000 7fe960f568c0 1 > prepared multipart upload key=R6ZDVOUE9NXJ upload=0KIMGN1H0U87 tag=9DNLD5U1ANNT type=write size=2228246 2024-03-04T17:50:37.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.748+0000 7fe960f568c0 1 < completed write operation key=ASC735ZCBLCG tag=IY2TJUHCPR8B type=write size=761263 2024-03-04T17:50:37.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.752+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=I8EOKHWW1SX9 type=write size=834104 2024-03-04T17:50:37.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.756+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~Z67EMXINSX3X.0 size=367350 2024-03-04T17:50:37.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.764+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=Z67EMXINSX3X tag=T59C6K49F4UI type=write size=1415926 2024-03-04T17:50:37.767 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.764+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=Y77F598HL1HS type=del size=2670039 2024-03-04T17:50:37.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.772+0000 7fe960f568c0 1 < suggested remove operation key=UEVMIPQ9FR81 tag=16QF04JR23KM type=del 2024-03-04T17:50:37.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.780+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~IFWGZS0FWJ32.0 size=646421 2024-03-04T17:50:37.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.780+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=IFWGZS0FWJ32 tag=2SFV8922CPDT type=write size=1694997 2024-03-04T17:50:37.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.784+0000 7fe960f568c0 1 < completed write operation key=ASC735ZCBLCG tag=12B7MD774B9Q type=write size=124116 2024-03-04T17:50:37.788 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.788+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=S8P1EN1CLFNP type=write size=1033482 2024-03-04T17:50:37.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.800+0000 7fe960f568c0 1 < suggested update operation key=QFF67JIW8Y8E tag=JA6WQVL5ZR9A type=write size=892643 2024-03-04T17:50:37.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.800+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=K6RYNIYKN7B1 type=write size=888848 2024-03-04T17:50:37.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.804+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=L036AX4IDG7E type=del 2024-03-04T17:50:37.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.808+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=7FJXPWMJPN29 type=del size=3281242 2024-03-04T17:50:37.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.812+0000 7fe960f568c0 1 < completed part key=_multipart_JT02WL9JYTNM.2~5C7DV4N72DO9.0 size=511711 2024-03-04T17:50:37.817 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.816+0000 7fe960f568c0 1 < completed multipart upload key=JT02WL9JYTNM upload=5C7DV4N72DO9 tag=20P00D6XYN08 type=write size=1560287 2024-03-04T17:50:37.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.824+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~SVK5OSTLM7HO.0 size=243141 2024-03-04T17:50:37.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.824+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=SVK5OSTLM7HO tag=HMRBCSZ2K3BV type=write size=1291717 2024-03-04T17:50:37.828 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.828+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=FPE1N2JH3IU1 type=del 2024-03-04T17:50:37.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.832+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=0092Z2R6HGCG type=del size=1378127 2024-03-04T17:50:37.836 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.836+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=G6KD0UR2N5Y7 type=del 2024-03-04T17:50:37.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.840+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=M0XVSLQX433N type=del size=2820101 2024-03-04T17:50:37.844 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.844+0000 7fe960f568c0 1 < completed write operation key=XSQ3GE2UHMZG tag=OQZLKDFACFZL type=write size=717267 2024-03-04T17:50:37.847 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.848+0000 7fe960f568c0 1 > prepared operation key=JP3FRB6MD7B4 tag=FIVEO6CD2DQL type=del size=2021750 2024-03-04T17:50:37.851 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.852+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=C7Q8HZ19SDVZ type=del 2024-03-04T17:50:37.855 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.856+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=X7535AS2MPJR type=del size=2534035 2024-03-04T17:50:37.863 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.864+0000 7fe960f568c0 1 < suggested remove operation key=GZ4RRIR5CC26 tag=EHERKUXEMMF2 type=del 2024-03-04T17:50:37.867 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.864+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=WD5CE3NFL6PG type=write size=604140 2024-03-04T17:50:37.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.876+0000 7fe960f568c0 1 < suggested update operation key=6X4ZV3BFZNDL tag=0AE90UKBD145 type=del size=3733054 2024-03-04T17:50:37.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.880+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=EZ9ENYTTV07F type=del size=2231135 2024-03-04T17:50:37.886 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.884+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~ET9LMT892Q6X.0 size=1048576 2024-03-04T17:50:37.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.888+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~ET9LMT892Q6X.1 size=680948 2024-03-04T17:50:37.896 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.896+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=ET9LMT892Q6X tag=5ZL0ZYHXY9UZ type=write size=2778100 2024-03-04T17:50:37.900 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.900+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=V7MGLFYIRIF8 type=del size=3120323 2024-03-04T17:50:37.901 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.900+0000 7fe960f568c0 1 < canceled part key=_multipart_KYSLL05AUHAC.2~FZ5LWD78EOVT.0 size=1048576 2024-03-04T17:50:37.904 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.904+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~FZ5LWD78EOVT.1 size=115654 2024-03-04T17:50:37.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.912+0000 7fe960f568c0 1 < canceled multipart upload key=KYSLL05AUHAC upload=FZ5LWD78EOVT tag=RMCK0MI6ULCP type=write size=2212806 2024-03-04T17:50:37.916 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.916+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=ZSF21XC3GIEB type=write size=444547 2024-03-04T17:50:37.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.920+0000 7fe960f568c0 1 < completed part key=_multipart_R6ZDVOUE9NXJ.2~0KIMGN1H0U87.0 size=1048576 2024-03-04T17:50:37.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.920+0000 7fe960f568c0 1 < canceled part key=_multipart_R6ZDVOUE9NXJ.2~0KIMGN1H0U87.1 size=131094 2024-03-04T17:50:37.927 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.928+0000 7fe960f568c0 1 < canceled multipart upload key=R6ZDVOUE9NXJ upload=0KIMGN1H0U87 tag=9DNLD5U1ANNT type=write size=2228246 2024-03-04T17:50:37.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.928+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=Q4UNDRFAG68C type=del size=105861 2024-03-04T17:50:37.935 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.932+0000 7fe960f568c0 1 < completed write operation key=RII14ZGX2EL1 tag=I8EOKHWW1SX9 type=write size=834104 2024-03-04T17:50:37.938 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.936+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=FWDJVIVMYT9C type=del size=3847018 2024-03-04T17:50:37.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.944+0000 7fe960f568c0 1 < suggested remove operation key=XSQ3GE2UHMZG tag=Y77F598HL1HS type=del 2024-03-04T17:50:37.950 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.948+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=FG2TJ0YNC4DJ type=write size=684936 2024-03-04T17:50:37.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.952+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~IFWGZS0FWJ32.0 size=646421 2024-03-04T17:50:37.960 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.960+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=IFWGZS0FWJ32 tag=2SFV8922CPDT type=write size=1694997 2024-03-04T17:50:37.964 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.964+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=KA4HIZ7KR6K4 type=del size=1108855 2024-03-04T17:50:37.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.968+0000 7fe960f568c0 1 < completed write operation key=UKZ079SVZZKG tag=S8P1EN1CLFNP type=write size=1033482 2024-03-04T17:50:37.972 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.972+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=YPEEQ8MVRQ7J type=del size=1930244 2024-03-04T17:50:37.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.976+0000 7fe960f568c0 1 < completed write operation key=3X4719GHK3BI tag=K6RYNIYKN7B1 type=write size=888848 2024-03-04T17:50:37.983 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.980+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~X5T8485N8WRK.0 size=1048576 2024-03-04T17:50:37.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.984+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~X5T8485N8WRK.1 size=362867 2024-03-04T17:50:37.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.984+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=X5T8485N8WRK tag=OZVGT75GBXQV type=write size=2460019 2024-03-04T17:50:37.988 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.988+0000 7fe960f568c0 -1 < no bi entry to suggest for operation key=XSQ3GE2UHMZG tag=7FJXPWMJPN29 type=del size=3281242: (2) No such file or directory 2024-03-04T17:50:37.995 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.996+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~1ZWYLLJ1VF0L.0 size=44261 2024-03-04T17:50:37.995 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:37.996+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=1ZWYLLJ1VF0L tag=GTQ13GEOID82 type=write size=1092837 2024-03-04T17:50:38.000 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.000+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~SVK5OSTLM7HO.0 size=243141 2024-03-04T17:50:38.007 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.004+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=SVK5OSTLM7HO tag=HMRBCSZ2K3BV type=write size=1291717 2024-03-04T17:50:38.010 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.008+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=IID32B99ACAC type=del size=538388 2024-03-04T17:50:38.018 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.016+0000 7fe960f568c0 1 < suggested update operation key=FLYDZHOIWO6L tag=0092Z2R6HGCG type=del size=1378127 2024-03-04T17:50:38.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.020+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=Y9MAAWAAHMGW type=write size=45301 2024-03-04T17:50:38.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.028+0000 7fe960f568c0 1 < suggested update operation key=P28N6U79ZUV2 tag=M0XVSLQX433N type=del size=2820101 2024-03-04T17:50:38.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.032+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=HZY6ZSH7YM1F type=del size=31666 2024-03-04T17:50:38.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.036+0000 7fe960f568c0 1 < completed delete operation key=JP3FRB6MD7B4 tag=FIVEO6CD2DQL type=del 2024-03-04T17:50:38.043 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.044+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~VZ66AO7PODEQ.0 size=1048576 2024-03-04T17:50:38.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.044+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~VZ66AO7PODEQ.1 size=824794 2024-03-04T17:50:38.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.044+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=VZ66AO7PODEQ tag=J0AUXCJIZ3Q2 type=write size=2921946 2024-03-04T17:50:38.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.048+0000 7fe960f568c0 -1 < failed to complete operation key=FLYDZHOIWO6L tag=X7535AS2MPJR type=del size=2534035: (22) Invalid argument 2024-03-04T17:50:38.056 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.056+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~2MCE1WVO75W4.0 size=1048576 2024-03-04T17:50:38.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.056+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~2MCE1WVO75W4.1 size=536299 2024-03-04T17:50:38.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.056+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=2MCE1WVO75W4 tag=4T41ER3M210K type=write size=2633451 2024-03-04T17:50:38.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.060+0000 7fe960f568c0 1 < completed write operation key=3YTAYUQLLUKY tag=WD5CE3NFL6PG type=write size=604140 2024-03-04T17:50:38.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.064+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=PS1QVCJSDI3V type=del size=227047 2024-03-04T17:50:38.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.068+0000 7fe960f568c0 1 < canceled operation key=UEVMIPQ9FR81 tag=EZ9ENYTTV07F type=del size=2231135 2024-03-04T17:50:38.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.072+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=JIZ5VYPL39KJ type=del size=316044 2024-03-04T17:50:38.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.076+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=V7MGLFYIRIF8 type=del 2024-03-04T17:50:38.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.084+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~BZ0MQXGAXXFL.0 size=1048576 2024-03-04T17:50:38.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.088+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~BZ0MQXGAXXFL.1 size=234376 2024-03-04T17:50:38.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.088+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=BZ0MQXGAXXFL tag=X2UJUJ93CLMM type=write size=2331528 2024-03-04T17:50:38.091 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.092+0000 7fe960f568c0 1 < completed write operation key=JT02WL9JYTNM tag=ZSF21XC3GIEB type=write size=444547 2024-03-04T17:50:38.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.096+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=3L1W7NNLBI2C type=del size=3982703 2024-03-04T17:50:38.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.100+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=Q4UNDRFAG68C type=del 2024-03-04T17:50:38.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.104+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~KGXD1V1T2GND.0 size=149916 2024-03-04T17:50:38.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.104+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=KGXD1V1T2GND tag=RBYB8ZFUO06D type=write size=1198492 2024-03-04T17:50:38.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.108+0000 7fe960f568c0 1 < canceled operation key=3X4719GHK3BI tag=FWDJVIVMYT9C type=del size=3847018 2024-03-04T17:50:38.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.116+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~W2NHB0HO9LQS.0 size=869755 2024-03-04T17:50:38.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.116+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=W2NHB0HO9LQS tag=V0345LRN7H38 type=write size=1918331 2024-03-04T17:50:38.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.120+0000 7fe960f568c0 1 < completed write operation key=2FUTMG0SSXBY tag=FG2TJ0YNC4DJ type=write size=684936 2024-03-04T17:50:38.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.124+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=JZX5Z9ET4YYS type=del size=429336 2024-03-04T17:50:38.127 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.124+0000 7fe960f568c0 1 < canceled operation key=78N8HHC4O4CG tag=KA4HIZ7KR6K4 type=del size=1108855 2024-03-04T17:50:38.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.128+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=N4EW6WJLOJ1T type=del size=729754 2024-03-04T17:50:38.134 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.132+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=YPEEQ8MVRQ7J type=del 2024-03-04T17:50:38.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.140+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~FK7O9Z9HABEE.0 size=1048576 2024-03-04T17:50:38.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.140+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~FK7O9Z9HABEE.1 size=360051 2024-03-04T17:50:38.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.140+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=FK7O9Z9HABEE tag=0U8YRVC5NUTD type=write size=2457203 2024-03-04T17:50:38.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.144+0000 7fe960f568c0 1 < canceled part key=_multipart_UKZ079SVZZKG.2~X5T8485N8WRK.0 size=1048576 2024-03-04T17:50:38.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.144+0000 7fe960f568c0 1 < canceled part key=_multipart_UKZ079SVZZKG.2~X5T8485N8WRK.1 size=362867 2024-03-04T17:50:38.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.148+0000 7fe960f568c0 1 < canceled multipart upload key=UKZ079SVZZKG upload=X5T8485N8WRK tag=OZVGT75GBXQV type=write size=2460019 2024-03-04T17:50:38.153 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.152+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=NGHVUYWF8EBX type=del size=2157752 2024-03-04T17:50:38.156 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.156+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~1ZWYLLJ1VF0L.0 size=44261 2024-03-04T17:50:38.162 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.160+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=1ZWYLLJ1VF0L tag=GTQ13GEOID82 type=write size=1092837 2024-03-04T17:50:38.166 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.164+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=NBIKUHOZZVAG type=write size=539263 2024-03-04T17:50:38.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.168+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=IID32B99ACAC type=del 2024-03-04T17:50:38.174 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.172+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=OGO6GU0XLHH3 type=del size=2037220 2024-03-04T17:50:38.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.176+0000 7fe960f568c0 1 < completed write operation key=JT02WL9JYTNM tag=Y9MAAWAAHMGW type=write size=45301 2024-03-04T17:50:38.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.184+0000 7fe960f568c0 1 > prepared part key=_multipart_NF4OIJWJ83DE.2~OBJYYSZG53ZZ.0 size=726762 2024-03-04T17:50:38.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.184+0000 7fe960f568c0 1 > prepared multipart upload key=NF4OIJWJ83DE upload=OBJYYSZG53ZZ tag=FBL0XFG81DYY type=write size=1775338 2024-03-04T17:50:38.190 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.188+0000 7fe960f568c0 1 < canceled operation key=3YTAYUQLLUKY tag=HZY6ZSH7YM1F type=del size=31666 2024-03-04T17:50:38.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.192+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=W62PLEBYOLIO type=del size=3173894 2024-03-04T17:50:38.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.196+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~VZ66AO7PODEQ.0 size=1048576 2024-03-04T17:50:38.201 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.200+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~VZ66AO7PODEQ.1 size=824794 2024-03-04T17:50:38.207 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.208+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=VZ66AO7PODEQ tag=J0AUXCJIZ3Q2 type=write size=2921946 2024-03-04T17:50:38.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.212+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=C1DAEMXEHTJV type=del size=3281264 2024-03-04T17:50:38.215 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.216+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~2MCE1WVO75W4.0 size=1048576 2024-03-04T17:50:38.219 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.216+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~2MCE1WVO75W4.1 size=536299 2024-03-04T17:50:38.225 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.224+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=2MCE1WVO75W4 tag=4T41ER3M210K type=write size=2633451 2024-03-04T17:50:38.232 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.232+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~ERM3985H29WE.0 size=1048576 2024-03-04T17:50:38.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.236+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~ERM3985H29WE.1 size=1048576 2024-03-04T17:50:38.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.240+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~ERM3985H29WE.2 size=366464 2024-03-04T17:50:38.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.240+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=ERM3985H29WE tag=F6ZEWEFLIGZM type=write size=3512192 2024-03-04T17:50:38.251 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.248+0000 7fe960f568c0 1 < suggested remove operation key=2FUTMG0SSXBY tag=PS1QVCJSDI3V type=del 2024-03-04T17:50:38.256 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.256+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=GT4YLNCHUASY type=write size=360991 2024-03-04T17:50:38.260 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.260+0000 7fe960f568c0 1 < completed delete operation key=MIZRUP4MEA0J tag=JIZ5VYPL39KJ type=del 2024-03-04T17:50:38.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.264+0000 7fe960f568c0 1 > prepared operation key=JT02WL9JYTNM tag=OOLJOB7FPDQA type=del size=613665 2024-03-04T17:50:38.270 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.268+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~BZ0MQXGAXXFL.0 size=1048576 2024-03-04T17:50:38.273 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.272+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~BZ0MQXGAXXFL.1 size=234376 2024-03-04T17:50:38.279 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.280+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=BZ0MQXGAXXFL tag=X2UJUJ93CLMM type=write size=2331528 2024-03-04T17:50:38.283 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.284+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=NSDVXSHQEPPM type=del size=2300370 2024-03-04T17:50:38.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.288+0000 7fe960f568c0 -1 < failed to complete operation key=2FUTMG0SSXBY tag=3L1W7NNLBI2C type=del size=3982703: (22) Invalid argument 2024-03-04T17:50:38.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.292+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~LURPZ9ONGFIJ.0 size=176491 2024-03-04T17:50:38.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.292+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=LURPZ9ONGFIJ tag=59IPOJXXUH5T type=write size=1225067 2024-03-04T17:50:38.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.296+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~KGXD1V1T2GND.0 size=149916 2024-03-04T17:50:38.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.304+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=KGXD1V1T2GND tag=RBYB8ZFUO06D type=write size=1198492 2024-03-04T17:50:38.310 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.308+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=BRZBY6OVI6H3 type=write size=183408 2024-03-04T17:50:38.314 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.312+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~W2NHB0HO9LQS.0 size=869755 2024-03-04T17:50:38.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.320+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=W2NHB0HO9LQS tag=V0345LRN7H38 type=write size=1918331 2024-03-04T17:50:38.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.328+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~KDEXC3T8D0RU.0 size=1048576 2024-03-04T17:50:38.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.328+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~KDEXC3T8D0RU.1 size=554262 2024-03-04T17:50:38.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.328+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=KDEXC3T8D0RU tag=LJDT6ANHJI2R type=write size=2651414 2024-03-04T17:50:38.335 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.336+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=JZX5Z9ET4YYS type=del 2024-03-04T17:50:38.339 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.340+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=VXD33K69KCH8 type=write size=375183 2024-03-04T17:50:38.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.344+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=N4EW6WJLOJ1T type=del 2024-03-04T17:50:38.348 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.348+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=LL9MZ3QEYXEF type=del size=3633981 2024-03-04T17:50:38.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.352+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~FK7O9Z9HABEE.0 size=1048576 2024-03-04T17:50:38.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.356+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~FK7O9Z9HABEE.1 size=360051 2024-03-04T17:50:38.361 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.360+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=FK7O9Z9HABEE tag=0U8YRVC5NUTD type=write size=2457203 2024-03-04T17:50:38.365 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.364+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=CGYXK929MVLI type=del size=2382564 2024-03-04T17:50:38.369 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.368+0000 7fe960f568c0 1 < completed delete operation key=3YTAYUQLLUKY tag=NGHVUYWF8EBX type=del 2024-03-04T17:50:38.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.376+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~PUYG4P13WZV4.0 size=67920 2024-03-04T17:50:38.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.376+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=PUYG4P13WZV4 tag=ALF23OTP6QU3 type=write size=1116496 2024-03-04T17:50:38.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.384+0000 7fe960f568c0 1 < suggested update operation key=P28N6U79ZUV2 tag=NBIKUHOZZVAG type=write size=539263 2024-03-04T17:50:38.390 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.388+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~YCQ7S7U7ZVMH.0 size=1048576 2024-03-04T17:50:38.393 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.392+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~YCQ7S7U7ZVMH.1 size=1048576 2024-03-04T17:50:38.396 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.396+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~YCQ7S7U7ZVMH.2 size=380367 2024-03-04T17:50:38.396 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.396+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=YCQ7S7U7ZVMH tag=5X9M1O8WBJIQ type=write size=3526095 2024-03-04T17:50:38.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.400+0000 7fe960f568c0 1 < completed delete operation key=WNXVI6K5MU3R tag=OGO6GU0XLHH3 type=del 2024-03-04T17:50:38.407 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.404+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~EC1FCO1MI5XN.0 size=1048576 2024-03-04T17:50:38.410 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.408+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~EC1FCO1MI5XN.1 size=1048576 2024-03-04T17:50:38.413 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.412+0000 7fe960f568c0 1 > prepared part key=_multipart_KYSLL05AUHAC.2~EC1FCO1MI5XN.2 size=297654 2024-03-04T17:50:38.413 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.412+0000 7fe960f568c0 1 > prepared multipart upload key=KYSLL05AUHAC upload=EC1FCO1MI5XN tag=FITJMK6X10ND type=write size=3443382 2024-03-04T17:50:38.414 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.412+0000 7fe960f568c0 1 < canceled part key=_multipart_NF4OIJWJ83DE.2~OBJYYSZG53ZZ.0 size=726762 2024-03-04T17:50:38.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.420+0000 7fe960f568c0 1 < canceled multipart upload key=NF4OIJWJ83DE upload=OBJYYSZG53ZZ tag=FBL0XFG81DYY type=write size=1775338 2024-03-04T17:50:38.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.424+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~08FWVGWPBS4F.0 size=1048576 2024-03-04T17:50:38.428 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.428+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~08FWVGWPBS4F.1 size=1048576 2024-03-04T17:50:38.431 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.432+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~08FWVGWPBS4F.2 size=381092 2024-03-04T17:50:38.431 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.432+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=08FWVGWPBS4F tag=CH5ZTWRB3DBG type=write size=3526820 2024-03-04T17:50:38.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.436+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=W62PLEBYOLIO type=del 2024-03-04T17:50:38.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.440+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=M30JTCU5PLX5 type=del size=1179548 2024-03-04T17:50:38.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.444+0000 7fe960f568c0 1 < completed delete operation key=BFF9OD00TU54 tag=C1DAEMXEHTJV type=del 2024-03-04T17:50:38.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.448+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~CLB8BGF8TZW3.0 size=1048576 2024-03-04T17:50:38.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.452+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~CLB8BGF8TZW3.1 size=354924 2024-03-04T17:50:38.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.452+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=CLB8BGF8TZW3 tag=0R54WQCL3GLY type=write size=2452076 2024-03-04T17:50:38.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.456+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~ERM3985H29WE.0 size=1048576 2024-03-04T17:50:38.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.456+0000 7fe960f568c0 1 < canceled part key=_multipart_FSWBX523X9V5.2~ERM3985H29WE.1 size=1048576 2024-03-04T17:50:38.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.456+0000 7fe960f568c0 1 < canceled part key=_multipart_FSWBX523X9V5.2~ERM3985H29WE.2 size=366464 2024-03-04T17:50:38.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.460+0000 7fe960f568c0 1 < canceled multipart upload key=FSWBX523X9V5 upload=ERM3985H29WE tag=F6ZEWEFLIGZM type=write size=3512192 2024-03-04T17:50:38.469 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.468+0000 7fe960f568c0 1 > prepared part key=_multipart_768X2XKWJW1S.2~95ZLD9RAV6J2.0 size=499849 2024-03-04T17:50:38.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.468+0000 7fe960f568c0 1 > prepared multipart upload key=768X2XKWJW1S upload=95ZLD9RAV6J2 tag=M1SWOQKKUV2U type=write size=1548425 2024-03-04T17:50:38.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.472+0000 7fe960f568c0 1 < completed write operation key=8PZD01OOX3AF tag=GT4YLNCHUASY type=write size=360991 2024-03-04T17:50:38.477 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.476+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=25LAGU1M9M01 type=del size=1369096 2024-03-04T17:50:38.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.480+0000 7fe960f568c0 1 < completed delete operation key=JT02WL9JYTNM tag=OOLJOB7FPDQA type=del 2024-03-04T17:50:38.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.484+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=NP3EC4KD113U type=write size=872654 2024-03-04T17:50:38.489 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.488+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=NSDVXSHQEPPM type=del 2024-03-04T17:50:38.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.492+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=F8O3Y01GTF4X type=del size=3657104 2024-03-04T17:50:38.497 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.496+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~LURPZ9ONGFIJ.0 size=176491 2024-03-04T17:50:38.503 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.500+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=LURPZ9ONGFIJ tag=59IPOJXXUH5T type=write size=1225067 2024-03-04T17:50:38.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.504+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=TDPM1SLKBA4N type=del size=3578423 2024-03-04T17:50:38.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.508+0000 7fe960f568c0 -1 < failed to complete operation key=P28N6U79ZUV2 tag=BRZBY6OVI6H3 type=write size=183408: (22) Invalid argument 2024-03-04T17:50:38.517 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.516+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~ER2PQVX11UJT.0 size=1048576 2024-03-04T17:50:38.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.520+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~ER2PQVX11UJT.1 size=309850 2024-03-04T17:50:38.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.520+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=ER2PQVX11UJT tag=PCD2H1IPAXNK type=write size=2407002 2024-03-04T17:50:38.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.524+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~KDEXC3T8D0RU.0 size=1048576 2024-03-04T17:50:38.527 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.528+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~KDEXC3T8D0RU.1 size=554262 2024-03-04T17:50:38.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.532+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=KDEXC3T8D0RU tag=LJDT6ANHJI2R type=write size=2651414 2024-03-04T17:50:38.538 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.536+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=QJT24GJZMIDT type=del size=4092436 2024-03-04T17:50:38.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.540+0000 7fe960f568c0 1 < completed write operation key=BFF9OD00TU54 tag=VXD33K69KCH8 type=write size=375183 2024-03-04T17:50:38.546 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.544+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=I0F6FU5TLZ4P type=del size=1645298 2024-03-04T17:50:38.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.548+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=LL9MZ3QEYXEF type=del 2024-03-04T17:50:38.554 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.552+0000 7fe960f568c0 1 > prepared operation key=UEVMIPQ9FR81 tag=ZZT0B5HX1FID type=del size=4068225 2024-03-04T17:50:38.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.556+0000 7fe960f568c0 1 < completed delete operation key=2MKQA5JYWU8L tag=CGYXK929MVLI type=del 2024-03-04T17:50:38.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.560+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=SVUSQ4BEFDMV type=del size=1089565 2024-03-04T17:50:38.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.568+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~PUYG4P13WZV4.0 size=67920 2024-03-04T17:50:38.574 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.572+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=PUYG4P13WZV4 tag=ALF23OTP6QU3 type=write size=1116496 2024-03-04T17:50:38.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.576+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=DAXAPX80NY9E type=del size=3362065 2024-03-04T17:50:38.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.580+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~YCQ7S7U7ZVMH.0 size=1048576 2024-03-04T17:50:38.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.584+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~YCQ7S7U7ZVMH.1 size=1048576 2024-03-04T17:50:38.588 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.588+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~YCQ7S7U7ZVMH.2 size=380367 2024-03-04T17:50:38.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.592+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=YCQ7S7U7ZVMH tag=5X9M1O8WBJIQ type=write size=3526095 2024-03-04T17:50:38.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.596+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=RIU4SP3YDSXJ type=write size=230474 2024-03-04T17:50:38.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.600+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~EC1FCO1MI5XN.0 size=1048576 2024-03-04T17:50:38.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.604+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~EC1FCO1MI5XN.1 size=1048576 2024-03-04T17:50:38.609 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.608+0000 7fe960f568c0 1 < completed part key=_multipart_KYSLL05AUHAC.2~EC1FCO1MI5XN.2 size=297654 2024-03-04T17:50:38.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.616+0000 7fe960f568c0 1 < completed multipart upload key=KYSLL05AUHAC upload=EC1FCO1MI5XN tag=FITJMK6X10ND type=write size=3443382 2024-03-04T17:50:38.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.620+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=ILS3F5YGENKU type=del size=3354321 2024-03-04T17:50:38.624 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.624+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~08FWVGWPBS4F.0 size=1048576 2024-03-04T17:50:38.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.624+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~08FWVGWPBS4F.1 size=1048576 2024-03-04T17:50:38.630 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.628+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~08FWVGWPBS4F.2 size=381092 2024-03-04T17:50:38.635 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.636+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=08FWVGWPBS4F tag=CH5ZTWRB3DBG type=write size=3526820 2024-03-04T17:50:38.639 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.640+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=W5TWC4RXBMYI type=write size=511892 2024-03-04T17:50:38.646 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.644+0000 7fe960f568c0 1 < suggested update operation key=XSQ3GE2UHMZG tag=M30JTCU5PLX5 type=del size=1179548 2024-03-04T17:50:38.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.648+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=EZOCEQKZ5O8X type=write size=318321 2024-03-04T17:50:38.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.648+0000 7fe960f568c0 1 < canceled part key=_multipart_JJ7SM1Y73A4C.2~CLB8BGF8TZW3.0 size=1048576 2024-03-04T17:50:38.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.652+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~CLB8BGF8TZW3.1 size=354924 2024-03-04T17:50:38.660 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.660+0000 7fe960f568c0 1 < canceled multipart upload key=JJ7SM1Y73A4C upload=CLB8BGF8TZW3 tag=0R54WQCL3GLY type=write size=2452076 2024-03-04T17:50:38.663 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.664+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=1QOI07OX60AZ type=del size=2807455 2024-03-04T17:50:38.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.664+0000 7fe960f568c0 1 < completed part key=_multipart_768X2XKWJW1S.2~95ZLD9RAV6J2.0 size=499849 2024-03-04T17:50:38.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.672+0000 7fe960f568c0 1 < completed multipart upload key=768X2XKWJW1S upload=95ZLD9RAV6J2 tag=M1SWOQKKUV2U type=write size=1548425 2024-03-04T17:50:38.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.680+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~HIQMRGN1EPP2.0 size=1048576 2024-03-04T17:50:38.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.680+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~HIQMRGN1EPP2.1 size=604147 2024-03-04T17:50:38.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.680+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=HIQMRGN1EPP2 tag=WLQ9TSVJIQP8 type=write size=2701299 2024-03-04T17:50:38.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.688+0000 7fe960f568c0 1 < suggested remove operation key=QGLN27WH1N2O tag=25LAGU1M9M01 type=del 2024-03-04T17:50:38.693 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.692+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=815J4QAGSY21 type=del size=3862662 2024-03-04T17:50:38.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.696+0000 7fe960f568c0 1 < completed write operation key=FLYDZHOIWO6L tag=NP3EC4KD113U type=write size=872654 2024-03-04T17:50:38.701 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.700+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=ABMWFU9J416U type=write size=895154 2024-03-04T17:50:38.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.704+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=F8O3Y01GTF4X type=del 2024-03-04T17:50:38.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.708+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=JCFNQ20NT2N3 type=del size=3647718 2024-03-04T17:50:38.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.716+0000 7fe960f568c0 1 < suggested remove operation key=UEVMIPQ9FR81 tag=TDPM1SLKBA4N type=del 2024-03-04T17:50:38.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.720+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=L5OEEA52Z365 type=del size=3123422 2024-03-04T17:50:38.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.724+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~ER2PQVX11UJT.0 size=1048576 2024-03-04T17:50:38.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.728+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~ER2PQVX11UJT.1 size=309850 2024-03-04T17:50:38.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.732+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=ER2PQVX11UJT tag=PCD2H1IPAXNK type=write size=2407002 2024-03-04T17:50:38.737 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.736+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=TNS1KJKKAN07 type=del size=48907 2024-03-04T17:50:38.740 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.740+0000 7fe960f568c0 1 < completed delete operation key=FLYDZHOIWO6L tag=QJT24GJZMIDT type=del 2024-03-04T17:50:38.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.744+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~UP9005P60PXS.0 size=46432 2024-03-04T17:50:38.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.744+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=UP9005P60PXS tag=FLS529FTEXFW type=write size=1095008 2024-03-04T17:50:38.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.748+0000 7fe960f568c0 -1 < no bi entry to suggest for operation key=UEVMIPQ9FR81 tag=I0F6FU5TLZ4P type=del size=1645298: (2) No such file or directory 2024-03-04T17:50:38.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.752+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=5CFPH5IT1UTE type=del size=556769 2024-03-04T17:50:38.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.756+0000 7fe960f568c0 -1 < failed to complete operation key=UEVMIPQ9FR81 tag=ZZT0B5HX1FID type=del size=4068225: (22) Invalid argument 2024-03-04T17:50:38.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.760+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~WZG1PZ4NA6R0.0 size=1048576 2024-03-04T17:50:38.765 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.764+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~WZG1PZ4NA6R0.1 size=271161 2024-03-04T17:50:38.765 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.764+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=WZG1PZ4NA6R0 tag=CX5Q1SJ3DWBJ type=write size=2368313 2024-03-04T17:50:38.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.768+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=SVUSQ4BEFDMV type=del 2024-03-04T17:50:38.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.772+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=P6TZ92T0EC03 type=del size=1313406 2024-03-04T17:50:38.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.776+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=DAXAPX80NY9E type=del 2024-03-04T17:50:38.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.784+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~WP0QZLV2T5VO.0 size=517384 2024-03-04T17:50:38.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.784+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=WP0QZLV2T5VO tag=JMY9ESSQAG0H type=write size=1565960 2024-03-04T17:50:38.788 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.788+0000 7fe960f568c0 1 < completed write operation key=3X4719GHK3BI tag=RIU4SP3YDSXJ type=write size=230474 2024-03-04T17:50:38.792 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.792+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=UWN37EJQZOKR type=del size=681970 2024-03-04T17:50:38.797 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.796+0000 7fe960f568c0 1 < completed delete operation key=2FUTMG0SSXBY tag=ILS3F5YGENKU type=del 2024-03-04T17:50:38.801 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.800+0000 7fe960f568c0 1 > prepared operation key=2MKQA5JYWU8L tag=WDZJHPJXNWUF type=write size=862395 2024-03-04T17:50:38.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.804+0000 7fe960f568c0 1 < completed write operation key=6X4ZV3BFZNDL tag=W5TWC4RXBMYI type=write size=511892 2024-03-04T17:50:38.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.812+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~EF5A1H364WZD.0 size=1048576 2024-03-04T17:50:38.814 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.812+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~EF5A1H364WZD.1 size=1048576 2024-03-04T17:50:38.818 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.816+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~EF5A1H364WZD.2 size=509071 2024-03-04T17:50:38.818 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.816+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=EF5A1H364WZD tag=7Z9MHWMDHAD4 type=write size=3654799 2024-03-04T17:50:38.821 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.820+0000 7fe960f568c0 1 < completed write operation key=JSO6WIYMPCF6 tag=EZOCEQKZ5O8X type=write size=318321 2024-03-04T17:50:38.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.824+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=KVC776M3V9ED type=del size=3554894 2024-03-04T17:50:38.829 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.828+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=1QOI07OX60AZ type=del 2024-03-04T17:50:38.833 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.832+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=KOP2PT3W5K6Q type=write size=997035 2024-03-04T17:50:38.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.836+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~HIQMRGN1EPP2.0 size=1048576 2024-03-04T17:50:38.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.840+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~HIQMRGN1EPP2.1 size=604147 2024-03-04T17:50:38.847 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.844+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=HIQMRGN1EPP2 tag=WLQ9TSVJIQP8 type=write size=2701299 2024-03-04T17:50:38.851 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.848+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=XI2WRFV1GX93 type=write size=26412 2024-03-04T17:50:38.855 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.852+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=815J4QAGSY21 type=del 2024-03-04T17:50:38.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.860+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~3Q5OXO0HV3RM.0 size=282899 2024-03-04T17:50:38.862 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.860+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=3Q5OXO0HV3RM tag=CAELIMX7CXA8 type=write size=1331475 2024-03-04T17:50:38.866 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.864+0000 7fe960f568c0 1 < completed write operation key=QGLN27WH1N2O tag=ABMWFU9J416U type=write size=895154 2024-03-04T17:50:38.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.868+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=OB22DGYKWWLC type=write size=838003 2024-03-04T17:50:38.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.872+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=JCFNQ20NT2N3 type=del 2024-03-04T17:50:38.877 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.876+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=9G3KLUIDNY10 type=del size=2785948 2024-03-04T17:50:38.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.880+0000 7fe960f568c0 1 < completed delete operation key=UKZ079SVZZKG tag=L5OEEA52Z365 type=del 2024-03-04T17:50:38.886 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.884+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=HW3KLWWEJ5WP type=write size=796835 2024-03-04T17:50:38.891 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.888+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=TNS1KJKKAN07 type=del 2024-03-04T17:50:38.899 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.896+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~303ZUQT8BF6F.0 size=1048576 2024-03-04T17:50:38.902 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.900+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~303ZUQT8BF6F.1 size=1048576 2024-03-04T17:50:38.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.904+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~303ZUQT8BF6F.2 size=314535 2024-03-04T17:50:38.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.904+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=303ZUQT8BF6F tag=5BZJYPLZGM8V type=write size=3460263 2024-03-04T17:50:38.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.908+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~UP9005P60PXS.0 size=46432 2024-03-04T17:50:38.916 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.916+0000 7fe960f568c0 1 < completed multipart upload key=FLYDZHOIWO6L upload=UP9005P60PXS tag=FLS529FTEXFW type=write size=1095008 2024-03-04T17:50:38.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.920+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=AI13V5G4DXPH type=del size=560442 2024-03-04T17:50:38.924 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.924+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=5CFPH5IT1UTE type=del 2024-03-04T17:50:38.928 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.928+0000 7fe960f568c0 1 > prepared operation key=XSQ3GE2UHMZG tag=2B2FOSO4EMFF type=del size=3517314 2024-03-04T17:50:38.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.932+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~WZG1PZ4NA6R0.0 size=1048576 2024-03-04T17:50:38.935 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.936+0000 7fe960f568c0 1 < completed part key=_multipart_JJ7SM1Y73A4C.2~WZG1PZ4NA6R0.1 size=271161 2024-03-04T17:50:38.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.940+0000 7fe960f568c0 1 < completed multipart upload key=JJ7SM1Y73A4C upload=WZG1PZ4NA6R0 tag=CX5Q1SJ3DWBJ type=write size=2368313 2024-03-04T17:50:38.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.944+0000 7fe960f568c0 1 > prepared operation key=BFF9OD00TU54 tag=TZ2PEIZB5OYQ type=del size=319126 2024-03-04T17:50:38.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.952+0000 7fe960f568c0 1 < suggested remove operation key=6X4ZV3BFZNDL tag=P6TZ92T0EC03 type=del 2024-03-04T17:50:38.959 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.956+0000 7fe960f568c0 1 > prepared part key=_multipart_FSWBX523X9V5.2~8984IXI3UQJ7.0 size=90902 2024-03-04T17:50:38.959 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.956+0000 7fe960f568c0 1 > prepared multipart upload key=FSWBX523X9V5 upload=8984IXI3UQJ7 tag=OK4ALIPM2VMR type=write size=1139478 2024-03-04T17:50:38.960 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.960+0000 7fe960f568c0 1 < canceled part key=_multipart_JP3FRB6MD7B4.2~WP0QZLV2T5VO.0 size=517384 2024-03-04T17:50:38.969 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.968+0000 7fe960f568c0 1 < canceled multipart upload key=JP3FRB6MD7B4 upload=WP0QZLV2T5VO tag=JMY9ESSQAG0H type=write size=1565960 2024-03-04T17:50:38.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.972+0000 7fe960f568c0 1 > prepared operation key=FLYDZHOIWO6L tag=VYWE685WKD69 type=del size=2793368 2024-03-04T17:50:38.978 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.976+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=UWN37EJQZOKR type=del 2024-03-04T17:50:38.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.980+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=45P8N5PLN3D7 type=del size=2054725 2024-03-04T17:50:38.985 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.984+0000 7fe960f568c0 1 < completed write operation key=2MKQA5JYWU8L tag=WDZJHPJXNWUF type=write size=862395 2024-03-04T17:50:38.989 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.988+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=I217KMO8HHAV type=del size=1057897 2024-03-04T17:50:38.993 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.992+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~EF5A1H364WZD.0 size=1048576 2024-03-04T17:50:38.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:38.996+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~EF5A1H364WZD.1 size=1048576 2024-03-04T17:50:38.999 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.000+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~EF5A1H364WZD.2 size=509071 2024-03-04T17:50:39.006 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.004+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=EF5A1H364WZD tag=7Z9MHWMDHAD4 type=write size=3654799 2024-03-04T17:50:39.011 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.008+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=YZ1UDIS6N0IP type=del size=2441434 2024-03-04T17:50:39.015 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.016+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=KVC776M3V9ED type=del 2024-03-04T17:50:39.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.020+0000 7fe960f568c0 1 > prepared operation key=ASC735ZCBLCG tag=D5WUVCTQDH3E type=del size=1651618 2024-03-04T17:50:39.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.032+0000 7fe960f568c0 1 < suggested remove operation key=JSO6WIYMPCF6 tag=KOP2PT3W5K6Q type=write 2024-03-04T17:50:39.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.032+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=56BSWXMI3KJ4 type=del size=2994785 2024-03-04T17:50:39.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.036+0000 7fe960f568c0 1 < completed write operation key=768X2XKWJW1S tag=XI2WRFV1GX93 type=write size=26412 2024-03-04T17:50:39.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.044+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~QYXZR8QLCFUY.0 size=1048576 2024-03-04T17:50:39.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.048+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~QYXZR8QLCFUY.1 size=1048576 2024-03-04T17:50:39.052 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.052+0000 7fe960f568c0 1 > prepared part key=_multipart_JP3FRB6MD7B4.2~QYXZR8QLCFUY.2 size=412487 2024-03-04T17:50:39.052 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.052+0000 7fe960f568c0 1 > prepared multipart upload key=JP3FRB6MD7B4 upload=QYXZR8QLCFUY tag=B3NPS72EE5WW type=write size=3558215 2024-03-04T17:50:39.055 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.056+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~3Q5OXO0HV3RM.0 size=282899 2024-03-04T17:50:39.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.060+0000 7fe960f568c0 1 < completed multipart upload key=XSQ3GE2UHMZG upload=3Q5OXO0HV3RM tag=CAELIMX7CXA8 type=write size=1331475 2024-03-04T17:50:39.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.064+0000 7fe960f568c0 1 > prepared operation key=QGLN27WH1N2O tag=0KJBFLSK37KM type=del size=2785078 2024-03-04T17:50:39.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.072+0000 7fe960f568c0 1 < suggested remove operation key=6X4ZV3BFZNDL tag=OB22DGYKWWLC type=write 2024-03-04T17:50:39.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.076+0000 7fe960f568c0 1 > prepared operation key=2FUTMG0SSXBY tag=HKHL1WAIH83D type=write size=923599 2024-03-04T17:50:39.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.080+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=9G3KLUIDNY10 type=del 2024-03-04T17:50:39.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.084+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=KDC1XFQ33H85 type=write size=736348 2024-03-04T17:50:39.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.088+0000 7fe960f568c0 1 < completed write operation key=U2G6YEVD5506 tag=HW3KLWWEJ5WP type=write size=796835 2024-03-04T17:50:39.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.092+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=43HHELPR5IKZ type=del size=1012082 2024-03-04T17:50:39.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.096+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~303ZUQT8BF6F.0 size=1048576 2024-03-04T17:50:39.101 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.100+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~303ZUQT8BF6F.1 size=1048576 2024-03-04T17:50:39.104 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.104+0000 7fe960f568c0 1 < completed part key=_multipart_P28N6U79ZUV2.2~303ZUQT8BF6F.2 size=314535 2024-03-04T17:50:39.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.108+0000 7fe960f568c0 1 < completed multipart upload key=P28N6U79ZUV2 upload=303ZUQT8BF6F tag=5BZJYPLZGM8V type=write size=3460263 2024-03-04T17:50:39.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.116+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~L8SCQW5GNFRB.0 size=518028 2024-03-04T17:50:39.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.116+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=L8SCQW5GNFRB tag=DSUWAV4PI6PM type=write size=1566604 2024-03-04T17:50:39.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.124+0000 7fe960f568c0 1 < suggested remove operation key=JJ7SM1Y73A4C tag=AI13V5G4DXPH type=del 2024-03-04T17:50:39.134 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.132+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~37J1R9RCYLQ3.0 size=1048576 2024-03-04T17:50:39.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.136+0000 7fe960f568c0 1 > prepared part key=_multipart_3X4719GHK3BI.2~37J1R9RCYLQ3.1 size=1017002 2024-03-04T17:50:39.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.136+0000 7fe960f568c0 1 > prepared multipart upload key=3X4719GHK3BI upload=37J1R9RCYLQ3 tag=SE9BI44KW1M2 type=write size=3114154 2024-03-04T17:50:39.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.140+0000 7fe960f568c0 1 < completed delete operation key=XSQ3GE2UHMZG tag=2B2FOSO4EMFF type=del 2024-03-04T17:50:39.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.148+0000 7fe960f568c0 1 > prepared part key=_multipart_6X4ZV3BFZNDL.2~9C7G4V1NTQNQ.0 size=176447 2024-03-04T17:50:39.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.148+0000 7fe960f568c0 1 > prepared multipart upload key=6X4ZV3BFZNDL upload=9C7G4V1NTQNQ tag=AK45JP28BGKM type=write size=1225023 2024-03-04T17:50:39.156 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.156+0000 7fe960f568c0 1 < suggested remove operation key=BFF9OD00TU54 tag=TZ2PEIZB5OYQ type=del 2024-03-04T17:50:39.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.160+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=APS4VNJ0Q5WO type=del size=454511 2024-03-04T17:50:39.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.164+0000 7fe960f568c0 1 < completed part key=_multipart_FSWBX523X9V5.2~8984IXI3UQJ7.0 size=90902 2024-03-04T17:50:39.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.168+0000 7fe960f568c0 1 < completed multipart upload key=FSWBX523X9V5 upload=8984IXI3UQJ7 tag=OK4ALIPM2VMR type=write size=1139478 2024-03-04T17:50:39.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.176+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~EFUGN6VF1PWA.0 size=1048576 2024-03-04T17:50:39.180 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.180+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~EFUGN6VF1PWA.1 size=1048576 2024-03-04T17:50:39.183 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.184+0000 7fe960f568c0 1 > prepared part key=_multipart_UKZ079SVZZKG.2~EFUGN6VF1PWA.2 size=102674 2024-03-04T17:50:39.183 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.184+0000 7fe960f568c0 1 > prepared multipart upload key=UKZ079SVZZKG upload=EFUGN6VF1PWA tag=CR036AKA05KX type=write size=3248402 2024-03-04T17:50:39.190 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.188+0000 7fe960f568c0 1 < suggested remove operation key=FLYDZHOIWO6L tag=VYWE685WKD69 type=del 2024-03-04T17:50:39.197 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.196+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~ISR5VYDW8VOT.0 size=1048576 2024-03-04T17:50:39.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.200+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~ISR5VYDW8VOT.1 size=1048576 2024-03-04T17:50:39.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.200+0000 7fe960f568c0 1 > prepared part key=_multipart_2KZ4IFW54C5B.2~ISR5VYDW8VOT.2 size=454108 2024-03-04T17:50:39.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.200+0000 7fe960f568c0 1 > prepared multipart upload key=2KZ4IFW54C5B upload=ISR5VYDW8VOT tag=N84B770FAKB5 type=write size=3599836 2024-03-04T17:50:39.206 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.204+0000 7fe960f568c0 -1 < failed to complete operation key=JJ7SM1Y73A4C tag=45P8N5PLN3D7 type=del size=2054725: (22) Invalid argument 2024-03-04T17:50:39.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.212+0000 7fe960f568c0 1 > prepared part key=_multipart_3YTAYUQLLUKY.2~AZE2UDNB3DMN.0 size=785852 2024-03-04T17:50:39.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.212+0000 7fe960f568c0 1 > prepared multipart upload key=3YTAYUQLLUKY upload=AZE2UDNB3DMN tag=NVPVI9BXMMB4 type=write size=1834428 2024-03-04T17:50:39.217 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.216+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=I217KMO8HHAV type=del 2024-03-04T17:50:39.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.224+0000 7fe960f568c0 1 > prepared part key=_multipart_JJ7SM1Y73A4C.2~0VQTZ1IZDWKI.0 size=22383 2024-03-04T17:50:39.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.224+0000 7fe960f568c0 1 > prepared multipart upload key=JJ7SM1Y73A4C upload=0VQTZ1IZDWKI tag=CZU6HIURKV5K type=write size=1070959 2024-03-04T17:50:39.227 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.228+0000 7fe960f568c0 -1 < failed to complete operation key=JJ7SM1Y73A4C tag=YZ1UDIS6N0IP type=del size=2441434: (22) Invalid argument 2024-03-04T17:50:39.233 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.232+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~R22XWSFF1APU.0 size=1048576 2024-03-04T17:50:39.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.236+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~R22XWSFF1APU.1 size=1048576 2024-03-04T17:50:39.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.236+0000 7fe960f568c0 1 > prepared part key=_multipart_JSO6WIYMPCF6.2~R22XWSFF1APU.2 size=172560 2024-03-04T17:50:39.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.236+0000 7fe960f568c0 1 > prepared multipart upload key=JSO6WIYMPCF6 upload=R22XWSFF1APU tag=9AAL21B471UI type=write size=3318288 2024-03-04T17:50:39.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.240+0000 7fe960f568c0 1 < completed delete operation key=ASC735ZCBLCG tag=D5WUVCTQDH3E type=del 2024-03-04T17:50:39.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.244+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=NTHE1C14JIS7 type=del size=1919464 2024-03-04T17:50:39.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.248+0000 7fe960f568c0 -1 < failed to complete operation key=6X4ZV3BFZNDL tag=56BSWXMI3KJ4 type=del size=2994785: (22) Invalid argument 2024-03-04T17:50:39.253 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.252+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=7JO2HUYPN9FG type=del size=1148591 2024-03-04T17:50:39.257 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.256+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~QYXZR8QLCFUY.0 size=1048576 2024-03-04T17:50:39.259 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.260+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~QYXZR8QLCFUY.1 size=1048576 2024-03-04T17:50:39.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.260+0000 7fe960f568c0 1 < completed part key=_multipart_JP3FRB6MD7B4.2~QYXZR8QLCFUY.2 size=412487 2024-03-04T17:50:39.268 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.268+0000 7fe960f568c0 1 < completed multipart upload key=JP3FRB6MD7B4 upload=QYXZR8QLCFUY tag=B3NPS72EE5WW type=write size=3558215 2024-03-04T17:50:39.275 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.272+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~JR1N6NZY43HM.0 size=1048576 2024-03-04T17:50:39.278 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.276+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~JR1N6NZY43HM.1 size=424076 2024-03-04T17:50:39.278 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.276+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=JR1N6NZY43HM tag=5DPV5VWIF1FH type=write size=2521228 2024-03-04T17:50:39.281 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.280+0000 7fe960f568c0 1 < canceled operation key=QGLN27WH1N2O tag=0KJBFLSK37KM type=del size=2785078 2024-03-04T17:50:39.285 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.284+0000 7fe960f568c0 1 > prepared operation key=768X2XKWJW1S tag=GFVI0DNG9217 type=del size=988161 2024-03-04T17:50:39.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.288+0000 7fe960f568c0 1 < completed write operation key=2FUTMG0SSXBY tag=HKHL1WAIH83D type=write size=923599 2024-03-04T17:50:39.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.292+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~SF9HQ72V2FKP.0 size=1048576 2024-03-04T17:50:39.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.296+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~SF9HQ72V2FKP.1 size=316481 2024-03-04T17:50:39.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.296+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=SF9HQ72V2FKP tag=P5V8MEV2LWN0 type=write size=2413633 2024-03-04T17:50:39.301 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.300+0000 7fe960f568c0 1 < completed write operation key=3X4719GHK3BI tag=KDC1XFQ33H85 type=write size=736348 2024-03-04T17:50:39.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.304+0000 7fe960f568c0 1 > prepared operation key=2KZ4IFW54C5B tag=QH2KRI7VGI49 type=del size=166155 2024-03-04T17:50:39.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.308+0000 7fe960f568c0 1 < completed delete operation key=3X4719GHK3BI tag=43HHELPR5IKZ type=del 2024-03-04T17:50:39.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.316+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~DPME3KJNEL2H.0 size=1048576 2024-03-04T17:50:39.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.320+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~DPME3KJNEL2H.1 size=1048576 2024-03-04T17:50:39.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.320+0000 7fe960f568c0 1 > prepared part key=_multipart_UEVMIPQ9FR81.2~DPME3KJNEL2H.2 size=47698 2024-03-04T17:50:39.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.320+0000 7fe960f568c0 1 > prepared multipart upload key=UEVMIPQ9FR81 upload=DPME3KJNEL2H tag=KNNXLWK2VYS2 type=write size=3193426 2024-03-04T17:50:39.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.328+0000 7fe960f568c0 1 < completed part key=_multipart_7Z7BPZ8XGBHZ.2~L8SCQW5GNFRB.0 size=518028 2024-03-04T17:50:39.336 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.336+0000 7fe960f568c0 1 < completed multipart upload key=7Z7BPZ8XGBHZ upload=L8SCQW5GNFRB tag=DSUWAV4PI6PM type=write size=1566604 2024-03-04T17:50:39.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.340+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=OB7I9S8YXRUD type=del size=229214 2024-03-04T17:50:39.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.344+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~37J1R9RCYLQ3.0 size=1048576 2024-03-04T17:50:39.348 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.348+0000 7fe960f568c0 1 < completed part key=_multipart_3X4719GHK3BI.2~37J1R9RCYLQ3.1 size=1017002 2024-03-04T17:50:39.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.352+0000 7fe960f568c0 1 < completed multipart upload key=3X4719GHK3BI upload=37J1R9RCYLQ3 tag=SE9BI44KW1M2 type=write size=3114154 2024-03-04T17:50:39.358 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.356+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=EILGW2IT1UGA type=del size=1139826 2024-03-04T17:50:39.361 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.360+0000 7fe960f568c0 1 < completed part key=_multipart_6X4ZV3BFZNDL.2~9C7G4V1NTQNQ.0 size=176447 2024-03-04T17:50:39.368 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.368+0000 7fe960f568c0 1 < completed multipart upload key=6X4ZV3BFZNDL upload=9C7G4V1NTQNQ tag=AK45JP28BGKM type=write size=1225023 2024-03-04T17:50:39.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.372+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~A0WGZIE4N1O1.0 size=165378 2024-03-04T17:50:39.375 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.372+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=A0WGZIE4N1O1 tag=CWHLQN9VV91R type=write size=1213954 2024-03-04T17:50:39.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.376+0000 7fe960f568c0 1 < completed delete operation key=JSO6WIYMPCF6 tag=APS4VNJ0Q5WO type=del 2024-03-04T17:50:39.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.380+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=EYGSCD2SQ2GS type=del size=1318460 2024-03-04T17:50:39.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.384+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~EFUGN6VF1PWA.0 size=1048576 2024-03-04T17:50:39.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.388+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~EFUGN6VF1PWA.1 size=1048576 2024-03-04T17:50:39.392 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.392+0000 7fe960f568c0 1 < completed part key=_multipart_UKZ079SVZZKG.2~EFUGN6VF1PWA.2 size=102674 2024-03-04T17:50:39.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.396+0000 7fe960f568c0 1 < completed multipart upload key=UKZ079SVZZKG upload=EFUGN6VF1PWA tag=CR036AKA05KX type=write size=3248402 2024-03-04T17:50:39.403 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.400+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=C6EY9Z1DRO5T type=del size=652400 2024-03-04T17:50:39.407 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.404+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~ISR5VYDW8VOT.0 size=1048576 2024-03-04T17:50:39.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.408+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~ISR5VYDW8VOT.1 size=1048576 2024-03-04T17:50:39.412 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.412+0000 7fe960f568c0 1 < completed part key=_multipart_2KZ4IFW54C5B.2~ISR5VYDW8VOT.2 size=454108 2024-03-04T17:50:39.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.416+0000 7fe960f568c0 1 < completed multipart upload key=2KZ4IFW54C5B upload=ISR5VYDW8VOT tag=N84B770FAKB5 type=write size=3599836 2024-03-04T17:50:39.422 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.420+0000 7fe960f568c0 1 > prepared operation key=JSO6WIYMPCF6 tag=A45L773DDQLU type=write size=1022527 2024-03-04T17:50:39.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.424+0000 7fe960f568c0 1 < completed part key=_multipart_3YTAYUQLLUKY.2~AZE2UDNB3DMN.0 size=785852 2024-03-04T17:50:39.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.432+0000 7fe960f568c0 1 < completed multipart upload key=3YTAYUQLLUKY upload=AZE2UDNB3DMN tag=NVPVI9BXMMB4 type=write size=1834428 2024-03-04T17:50:39.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.436+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~T8ED2FV2FSH0.0 size=116636 2024-03-04T17:50:39.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.436+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=T8ED2FV2FSH0 tag=H9QXKZ7G7E0I type=write size=1165212 2024-03-04T17:50:39.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.440+0000 7fe960f568c0 1 < canceled part key=_multipart_JJ7SM1Y73A4C.2~0VQTZ1IZDWKI.0 size=22383 2024-03-04T17:50:39.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.444+0000 7fe960f568c0 1 < canceled multipart upload key=JJ7SM1Y73A4C upload=0VQTZ1IZDWKI tag=CZU6HIURKV5K type=write size=1070959 2024-03-04T17:50:39.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.448+0000 7fe960f568c0 1 > prepared operation key=U2G6YEVD5506 tag=D67ONJXVOAAF type=del size=2232107 2024-03-04T17:50:39.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.452+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~R22XWSFF1APU.0 size=1048576 2024-03-04T17:50:39.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.456+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~R22XWSFF1APU.1 size=1048576 2024-03-04T17:50:39.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.456+0000 7fe960f568c0 1 < completed part key=_multipart_JSO6WIYMPCF6.2~R22XWSFF1APU.2 size=172560 2024-03-04T17:50:39.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.464+0000 7fe960f568c0 1 < completed multipart upload key=JSO6WIYMPCF6 upload=R22XWSFF1APU tag=9AAL21B471UI type=write size=3318288 2024-03-04T17:50:39.469 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.468+0000 7fe960f568c0 1 > prepared operation key=MIZRUP4MEA0J tag=QMNZ2PXKZ67A type=write size=943248 2024-03-04T17:50:39.472 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.472+0000 7fe960f568c0 1 < completed delete operation key=RII14ZGX2EL1 tag=NTHE1C14JIS7 type=del 2024-03-04T17:50:39.476 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.476+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=HZ47J4A4FHA3 type=del size=2872863 2024-03-04T17:50:39.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.480+0000 7fe960f568c0 1 < completed delete operation key=768X2XKWJW1S tag=7JO2HUYPN9FG type=del 2024-03-04T17:50:39.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.488+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~MSKMRPVSEOW8.0 size=1048576 2024-03-04T17:50:39.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.492+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~MSKMRPVSEOW8.1 size=1048576 2024-03-04T17:50:39.494 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.492+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~MSKMRPVSEOW8.2 size=599907 2024-03-04T17:50:39.494 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.492+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=MSKMRPVSEOW8 tag=YCF0YXHVUCDZ type=write size=3745635 2024-03-04T17:50:39.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.496+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~JR1N6NZY43HM.0 size=1048576 2024-03-04T17:50:39.502 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.500+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~JR1N6NZY43HM.1 size=424076 2024-03-04T17:50:39.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.508+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=JR1N6NZY43HM tag=5DPV5VWIF1FH type=write size=2521228 2024-03-04T17:50:39.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.512+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=E7RQXMZJEUW3 type=del size=2081896 2024-03-04T17:50:39.515 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.516+0000 7fe960f568c0 1 < canceled operation key=768X2XKWJW1S tag=GFVI0DNG9217 type=del size=988161 2024-03-04T17:50:39.519 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.520+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=KV9Y6WK4AX7T type=write size=880419 2024-03-04T17:50:39.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.520+0000 7fe960f568c0 1 < canceled part key=_multipart_8PZD01OOX3AF.2~SF9HQ72V2FKP.0 size=1048576 2024-03-04T17:50:39.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.520+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~SF9HQ72V2FKP.1 size=316481 2024-03-04T17:50:39.529 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.528+0000 7fe960f568c0 1 < canceled multipart upload key=8PZD01OOX3AF upload=SF9HQ72V2FKP tag=P5V8MEV2LWN0 type=write size=2413633 2024-03-04T17:50:39.535 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.536+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~HEKEY6OD3BE8.0 size=1048576 2024-03-04T17:50:39.538 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.536+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~HEKEY6OD3BE8.1 size=1048576 2024-03-04T17:50:39.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.540+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~HEKEY6OD3BE8.2 size=749966 2024-03-04T17:50:39.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.540+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=HEKEY6OD3BE8 tag=3IHLWT4I7MUA type=write size=3895694 2024-03-04T17:50:39.544 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.544+0000 7fe960f568c0 1 < completed delete operation key=2KZ4IFW54C5B tag=QH2KRI7VGI49 type=del 2024-03-04T17:50:39.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.548+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=86I1HL0NF08I type=del size=3322823 2024-03-04T17:50:39.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.552+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~DPME3KJNEL2H.0 size=1048576 2024-03-04T17:50:39.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.556+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~DPME3KJNEL2H.1 size=1048576 2024-03-04T17:50:39.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.556+0000 7fe960f568c0 1 < completed part key=_multipart_UEVMIPQ9FR81.2~DPME3KJNEL2H.2 size=47698 2024-03-04T17:50:39.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.564+0000 7fe960f568c0 1 < completed multipart upload key=UEVMIPQ9FR81 upload=DPME3KJNEL2H tag=KNNXLWK2VYS2 type=write size=3193426 2024-03-04T17:50:39.570 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.568+0000 7fe960f568c0 1 > prepared operation key=P28N6U79ZUV2 tag=NSG42ZLQIF1R type=del size=304631 2024-03-04T17:50:39.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.572+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=OB7I9S8YXRUD type=del 2024-03-04T17:50:39.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.576+0000 7fe960f568c0 1 > prepared operation key=JJ7SM1Y73A4C tag=PYJQ5GFIL9M8 type=del size=537948 2024-03-04T17:50:39.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.580+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=EILGW2IT1UGA type=del 2024-03-04T17:50:39.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.584+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=4O2I6FBZ10WW type=del size=1366662 2024-03-04T17:50:39.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.588+0000 7fe960f568c0 1 < completed part key=_multipart_ASC735ZCBLCG.2~A0WGZIE4N1O1.0 size=165378 2024-03-04T17:50:39.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.596+0000 7fe960f568c0 1 < completed multipart upload key=ASC735ZCBLCG upload=A0WGZIE4N1O1 tag=CWHLQN9VV91R type=write size=1213954 2024-03-04T17:50:39.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.600+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~9OFST88K0MTJ.0 size=1048576 2024-03-04T17:50:39.605 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.604+0000 7fe960f568c0 1 > prepared part key=_multipart_BFF9OD00TU54.2~9OFST88K0MTJ.1 size=120262 2024-03-04T17:50:39.605 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.604+0000 7fe960f568c0 1 > prepared multipart upload key=BFF9OD00TU54 upload=9OFST88K0MTJ tag=FZDDVZZZEEFL type=write size=2217414 2024-03-04T17:50:39.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.608+0000 7fe960f568c0 1 < canceled operation key=NF4OIJWJ83DE tag=EYGSCD2SQ2GS type=del size=1318460 2024-03-04T17:50:39.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.616+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~JZQSWAWJOYOV.0 size=1048576 2024-03-04T17:50:39.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.616+0000 7fe960f568c0 1 > prepared part key=_multipart_QFF67JIW8Y8E.2~JZQSWAWJOYOV.1 size=725789 2024-03-04T17:50:39.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.616+0000 7fe960f568c0 1 > prepared multipart upload key=QFF67JIW8Y8E upload=JZQSWAWJOYOV tag=XNSF4UDRPKQF type=write size=2822941 2024-03-04T17:50:39.621 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.620+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=C6EY9Z1DRO5T type=del 2024-03-04T17:50:39.625 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.624+0000 7fe960f568c0 1 > prepared operation key=3YTAYUQLLUKY tag=F5X933IZ5GYK type=write size=390596 2024-03-04T17:50:39.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.628+0000 7fe960f568c0 1 < canceled operation key=JSO6WIYMPCF6 tag=A45L773DDQLU type=write size=1022527 2024-03-04T17:50:39.636 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.636+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~VJF5D6W6E277.0 size=1048576 2024-03-04T17:50:39.639 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.636+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~VJF5D6W6E277.1 size=1048576 2024-03-04T17:50:39.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.640+0000 7fe960f568c0 1 > prepared part key=_multipart_78N8HHC4O4CG.2~VJF5D6W6E277.2 size=313868 2024-03-04T17:50:39.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.640+0000 7fe960f568c0 1 > prepared multipart upload key=78N8HHC4O4CG upload=VJF5D6W6E277 tag=UP6NKBSFTBUC type=write size=3459596 2024-03-04T17:50:39.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.644+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~T8ED2FV2FSH0.0 size=116636 2024-03-04T17:50:39.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.648+0000 7fe960f568c0 1 < completed multipart upload key=XSQ3GE2UHMZG upload=T8ED2FV2FSH0 tag=H9QXKZ7G7E0I type=write size=1165212 2024-03-04T17:50:39.658 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.656+0000 7fe960f568c0 1 > prepared part key=_multipart_8PZD01OOX3AF.2~K2Q1OQG5JMR1.0 size=1047345 2024-03-04T17:50:39.658 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.656+0000 7fe960f568c0 1 > prepared multipart upload key=8PZD01OOX3AF upload=K2Q1OQG5JMR1 tag=QY3T2QH2T8PQ type=write size=2095921 2024-03-04T17:50:39.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.664+0000 7fe960f568c0 1 < suggested update operation key=U2G6YEVD5506 tag=D67ONJXVOAAF type=del size=2232107 2024-03-04T17:50:39.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.668+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=8WHM6KV6U8M6 type=write size=90444 2024-03-04T17:50:39.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.676+0000 7fe960f568c0 1 < completed write operation key=MIZRUP4MEA0J tag=QMNZ2PXKZ67A type=write size=943248 2024-03-04T17:50:39.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.680+0000 7fe960f568c0 1 > prepared operation key=3X4719GHK3BI tag=T48ELXSD3HPP type=del size=1275462 2024-03-04T17:50:39.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.684+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=HZ47J4A4FHA3 type=del 2024-03-04T17:50:39.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.688+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=RFPZT8AKLXGD type=del size=3127800 2024-03-04T17:50:39.693 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.692+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~MSKMRPVSEOW8.0 size=1048576 2024-03-04T17:50:39.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.696+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~MSKMRPVSEOW8.1 size=1048576 2024-03-04T17:50:39.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.696+0000 7fe960f568c0 1 < completed part key=_multipart_RII14ZGX2EL1.2~MSKMRPVSEOW8.2 size=599907 2024-03-04T17:50:39.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.704+0000 7fe960f568c0 1 < completed multipart upload key=RII14ZGX2EL1 upload=MSKMRPVSEOW8 tag=YCF0YXHVUCDZ type=write size=3745635 2024-03-04T17:50:39.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.708+0000 7fe960f568c0 1 > prepared operation key=R6ZDVOUE9NXJ tag=X1BJ1CINYDCT type=del size=1974998 2024-03-04T17:50:39.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.712+0000 7fe960f568c0 1 < completed delete operation key=78N8HHC4O4CG tag=E7RQXMZJEUW3 type=del 2024-03-04T17:50:39.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.716+0000 7fe960f568c0 1 > prepared operation key=6X4ZV3BFZNDL tag=22DQX7GSY3OA type=del size=3298952 2024-03-04T17:50:39.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.720+0000 7fe960f568c0 1 < completed write operation key=8PZD01OOX3AF tag=KV9Y6WK4AX7T type=write size=880419 2024-03-04T17:50:39.726 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.724+0000 7fe960f568c0 1 > prepared operation key=NF4OIJWJ83DE tag=GV9CWAVGFJLW type=write size=820402 2024-03-04T17:50:39.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.724+0000 7fe960f568c0 1 < canceled part key=_multipart_FLYDZHOIWO6L.2~HEKEY6OD3BE8.0 size=1048576 2024-03-04T17:50:39.730 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.728+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~HEKEY6OD3BE8.1 size=1048576 2024-03-04T17:50:39.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.732+0000 7fe960f568c0 1 < completed part key=_multipart_FLYDZHOIWO6L.2~HEKEY6OD3BE8.2 size=749966 2024-03-04T17:50:39.739 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.740+0000 7fe960f568c0 1 < canceled multipart upload key=FLYDZHOIWO6L upload=HEKEY6OD3BE8 tag=3IHLWT4I7MUA type=write size=3895694 2024-03-04T17:50:39.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.744+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~8M0RHPMVYF4N.0 size=872684 2024-03-04T17:50:39.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.744+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=8M0RHPMVYF4N tag=1IT1N6S1Z6XL type=write size=1921260 2024-03-04T17:50:39.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.748+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=86I1HL0NF08I type=del 2024-03-04T17:50:39.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.756+0000 7fe960f568c0 1 > prepared part key=_multipart_WNXVI6K5MU3R.2~P42QB4Y1JGQM.0 size=386920 2024-03-04T17:50:39.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.756+0000 7fe960f568c0 1 > prepared multipart upload key=WNXVI6K5MU3R upload=P42QB4Y1JGQM tag=VP4E0FXMN0S6 type=write size=1435496 2024-03-04T17:50:39.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.760+0000 7fe960f568c0 1 < completed delete operation key=P28N6U79ZUV2 tag=NSG42ZLQIF1R type=del 2024-03-04T17:50:39.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.764+0000 7fe960f568c0 1 > prepared operation key=7Z7BPZ8XGBHZ tag=3INBK6WYQ5BU type=write size=306945 2024-03-04T17:50:39.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.768+0000 7fe960f568c0 1 < completed delete operation key=JJ7SM1Y73A4C tag=PYJQ5GFIL9M8 type=del 2024-03-04T17:50:39.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.776+0000 7fe960f568c0 1 > prepared operation key=GZ4RRIR5CC26 tag=HV5S61B9HLBC type=del size=1451186 2024-03-04T17:50:39.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.780+0000 7fe960f568c0 1 < completed delete operation key=FSWBX523X9V5 tag=4O2I6FBZ10WW type=del 2024-03-04T17:50:39.788 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.788+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~BTCDMAL2ARG0.0 size=1048576 2024-03-04T17:50:39.791 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.792+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~BTCDMAL2ARG0.1 size=1048576 2024-03-04T17:50:39.795 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.792+0000 7fe960f568c0 1 > prepared part key=_multipart_P28N6U79ZUV2.2~BTCDMAL2ARG0.2 size=815183 2024-03-04T17:50:39.795 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.792+0000 7fe960f568c0 1 > prepared multipart upload key=P28N6U79ZUV2 upload=BTCDMAL2ARG0 tag=PUVTNKL6MYGY type=write size=3960911 2024-03-04T17:50:39.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.800+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~9OFST88K0MTJ.0 size=1048576 2024-03-04T17:50:39.803 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.804+0000 7fe960f568c0 1 < completed part key=_multipart_BFF9OD00TU54.2~9OFST88K0MTJ.1 size=120262 2024-03-04T17:50:39.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.812+0000 7fe960f568c0 1 < completed multipart upload key=BFF9OD00TU54 upload=9OFST88K0MTJ tag=FZDDVZZZEEFL type=write size=2217414 2024-03-04T17:50:39.816 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.816+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=S84PQZTDRABZ type=del size=364713 2024-03-04T17:50:39.821 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.820+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~JZQSWAWJOYOV.0 size=1048576 2024-03-04T17:50:39.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.824+0000 7fe960f568c0 1 < completed part key=_multipart_QFF67JIW8Y8E.2~JZQSWAWJOYOV.1 size=725789 2024-03-04T17:50:39.830 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.828+0000 7fe960f568c0 1 < completed multipart upload key=QFF67JIW8Y8E upload=JZQSWAWJOYOV tag=XNSF4UDRPKQF type=write size=2822941 2024-03-04T17:50:39.834 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.832+0000 7fe960f568c0 1 > prepared operation key=78N8HHC4O4CG tag=9VENZ8B4V8NX type=write size=223077 2024-03-04T17:50:39.842 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.840+0000 7fe960f568c0 1 < suggested update operation key=3YTAYUQLLUKY tag=F5X933IZ5GYK type=write size=390596 2024-03-04T17:50:39.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.848+0000 7fe960f568c0 1 > prepared part key=_multipart_7Z7BPZ8XGBHZ.2~GLUH07VGMWEQ.0 size=197051 2024-03-04T17:50:39.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.848+0000 7fe960f568c0 1 > prepared multipart upload key=7Z7BPZ8XGBHZ upload=GLUH07VGMWEQ tag=A6JYEBNHH9YN type=write size=1245627 2024-03-04T17:50:39.853 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.852+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~VJF5D6W6E277.0 size=1048576 2024-03-04T17:50:39.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.856+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~VJF5D6W6E277.1 size=1048576 2024-03-04T17:50:39.859 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.860+0000 7fe960f568c0 1 < completed part key=_multipart_78N8HHC4O4CG.2~VJF5D6W6E277.2 size=313868 2024-03-04T17:50:39.865 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.864+0000 7fe960f568c0 1 < completed multipart upload key=78N8HHC4O4CG upload=VJF5D6W6E277 tag=UP6NKBSFTBUC type=write size=3459596 2024-03-04T17:50:39.869 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.868+0000 7fe960f568c0 1 > prepared operation key=FSWBX523X9V5 tag=G28Z6TMY3KFI type=write size=901963 2024-03-04T17:50:39.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.872+0000 7fe960f568c0 1 < completed part key=_multipart_8PZD01OOX3AF.2~K2Q1OQG5JMR1.0 size=1047345 2024-03-04T17:50:39.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.880+0000 7fe960f568c0 1 < completed multipart upload key=8PZD01OOX3AF upload=K2Q1OQG5JMR1 tag=QY3T2QH2T8PQ type=write size=2095921 2024-03-04T17:50:39.886 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.884+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~KGRISUL9UY1D.0 size=883436 2024-03-04T17:50:39.886 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.884+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=KGRISUL9UY1D tag=SLYW4UU2PWI9 type=write size=1932012 2024-03-04T17:50:39.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.888+0000 7fe960f568c0 1 < completed write operation key=7Z7BPZ8XGBHZ tag=8WHM6KV6U8M6 type=write size=90444 2024-03-04T17:50:39.894 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.892+0000 7fe960f568c0 1 > prepared operation key=RII14ZGX2EL1 tag=5LK55YOTPMJM type=del size=2632540 2024-03-04T17:50:39.902 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.900+0000 7fe960f568c0 1 < suggested remove operation key=3X4719GHK3BI tag=T48ELXSD3HPP type=del 2024-03-04T17:50:39.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.908+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~BC26Q9L0V57Q.0 size=520641 2024-03-04T17:50:39.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.908+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=BC26Q9L0V57Q tag=1ZM94SOKR62F type=write size=1569217 2024-03-04T17:50:39.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.912+0000 7fe960f568c0 1 < completed delete operation key=8PZD01OOX3AF tag=RFPZT8AKLXGD type=del 2024-03-04T17:50:39.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.920+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~MDIRHGXRJTZ2.0 size=1048576 2024-03-04T17:50:39.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.924+0000 7fe960f568c0 1 > prepared part key=_multipart_ASC735ZCBLCG.2~MDIRHGXRJTZ2.1 size=676523 2024-03-04T17:50:39.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.924+0000 7fe960f568c0 1 > prepared multipart upload key=ASC735ZCBLCG upload=MDIRHGXRJTZ2 tag=4IWOTWUAZPC7 type=write size=2773675 2024-03-04T17:50:39.927 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.924+0000 7fe960f568c0 1 < completed delete operation key=R6ZDVOUE9NXJ tag=X1BJ1CINYDCT type=del 2024-03-04T17:50:39.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.928+0000 7fe960f568c0 1 > prepared operation key=UKZ079SVZZKG tag=15MWFMTVOR8V type=del size=2877759 2024-03-04T17:50:39.935 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.932+0000 7fe960f568c0 1 < completed delete operation key=6X4ZV3BFZNDL tag=22DQX7GSY3OA type=del 2024-03-04T17:50:39.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.940+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~J7Q8US56MD5S.0 size=1048576 2024-03-04T17:50:39.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.944+0000 7fe960f568c0 1 > prepared part key=_multipart_XSQ3GE2UHMZG.2~J7Q8US56MD5S.1 size=345548 2024-03-04T17:50:39.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.944+0000 7fe960f568c0 1 > prepared multipart upload key=XSQ3GE2UHMZG upload=J7Q8US56MD5S tag=GEZFGKUGJRIU type=write size=2442700 2024-03-04T17:50:39.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.948+0000 7fe960f568c0 1 < canceled operation key=NF4OIJWJ83DE tag=GV9CWAVGFJLW type=write size=820402 2024-03-04T17:50:39.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.952+0000 7fe960f568c0 1 > prepared part key=_multipart_2MKQA5JYWU8L.2~KS1B2C31UX9F.0 size=79338 2024-03-04T17:50:39.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.952+0000 7fe960f568c0 1 > prepared multipart upload key=2MKQA5JYWU8L upload=KS1B2C31UX9F tag=B7BYX6ZGCC83 type=write size=1127914 2024-03-04T17:50:39.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.956+0000 7fe960f568c0 1 < completed part key=_multipart_XSQ3GE2UHMZG.2~8M0RHPMVYF4N.0 size=872684 2024-03-04T17:50:39.963 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.964+0000 7fe960f568c0 1 < completed multipart upload key=XSQ3GE2UHMZG upload=8M0RHPMVYF4N tag=1IT1N6S1Z6XL type=write size=1921260 2024-03-04T17:50:39.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.968+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~B2Y1EUYIDA6U.0 size=1048576 2024-03-04T17:50:39.973 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.972+0000 7fe960f568c0 1 > prepared part key=_multipart_RII14ZGX2EL1.2~B2Y1EUYIDA6U.1 size=128005 2024-03-04T17:50:39.973 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.972+0000 7fe960f568c0 1 > prepared multipart upload key=RII14ZGX2EL1 upload=B2Y1EUYIDA6U tag=TE8ECVT1B5CI type=write size=2225157 2024-03-04T17:50:39.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.972+0000 7fe960f568c0 1 < canceled part key=_multipart_WNXVI6K5MU3R.2~P42QB4Y1JGQM.0 size=386920 2024-03-04T17:50:39.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.980+0000 7fe960f568c0 1 < canceled multipart upload key=WNXVI6K5MU3R upload=P42QB4Y1JGQM tag=VP4E0FXMN0S6 type=write size=1435496 2024-03-04T17:50:39.987 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.984+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~ZHUL7CEE34XM.0 size=1048576 2024-03-04T17:50:39.990 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.988+0000 7fe960f568c0 1 > prepared part key=_multipart_FLYDZHOIWO6L.2~ZHUL7CEE34XM.1 size=317389 2024-03-04T17:50:39.990 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.988+0000 7fe960f568c0 1 > prepared multipart upload key=FLYDZHOIWO6L upload=ZHUL7CEE34XM tag=821PH2IPS2V0 type=write size=2414541 2024-03-04T17:50:39.994 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.992+0000 7fe960f568c0 1 < completed write operation key=7Z7BPZ8XGBHZ tag=3INBK6WYQ5BU type=write size=306945 2024-03-04T17:50:39.998 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:39.996+0000 7fe960f568c0 1 > prepared operation key=8PZD01OOX3AF tag=3J5SIILC756N type=del size=1456900 2024-03-04T17:50:40.003 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:40.004+0000 7fe960f568c0 1 < completed delete operation key=GZ4RRIR5CC26 tag=HV5S61B9HLBC type=del 2024-03-04T17:50:40.007 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:50:40.008+0000 7fe960f568c0 1 > prepared operation key=WNXVI6K5MU3R tag=PPOG5TGKMTAT type=del size=1059717 2024-03-04T17:50:40.008 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] cls_rgw_stats.simulate (27652 ms) 2024-03-04T17:50:40.008 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 1 test from cls_rgw_stats (27652 ms total) 2024-03-04T17:50:40.008 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:50:40.008 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:50:40.243 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 1 test from 1 test suite ran. (29778 ms total) 2024-03-04T17:50:40.243 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 1 test. 2024-03-04T17:50:40.249 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:50:40.249 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:50:40.304 INFO:tasks.workunit:Running workunits matching cls/test_cls_cmpomap.sh on client.0... 2024-03-04T17:50:40.306 INFO:tasks.workunit:Running workunit cls/test_cls_cmpomap.sh... 2024-03-04T17:50:40.306 DEBUG:teuthology.orchestra.run.smithi137:workunit test cls/test_cls_cmpomap.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_cmpomap.sh 2024-03-04T17:50:40.370 INFO:tasks.workunit.client.0.smithi137.stdout:Running main() from gmock_main.cc 2024-03-04T17:50:40.370 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 31 tests from 1 test suite. 2024-03-04T17:50:40.370 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:50:42.288 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 31 tests from CmpOmap 2024-03-04T17:50:42.288 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str 2024-03-04T17:50:42.295 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str (7 ms) 2024-03-04T17:50:42.295 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str_default 2024-03-04T17:50:42.300 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str_default (5 ms) 2024-03-04T17:50:42.300 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64 2024-03-04T17:50:42.305 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64 (5 ms) 2024-03-04T17:50:42.306 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64_default 2024-03-04T17:50:42.312 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64_default (6 ms) 2024-03-04T17:50:42.312 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_vals_str 2024-03-04T17:50:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_vals_str (10 ms) 2024-03-04T17:50:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_vals_u64 2024-03-04T17:50:42.336 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_vals_u64 (14 ms) 2024-03-04T17:50:42.337 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_input 2024-03-04T17:50:42.343 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_input (7 ms) 2024-03-04T17:50:42.343 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_default 2024-03-04T17:50:42.348 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_default (5 ms) 2024-03-04T17:50:42.348 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_vals_u64_empty_default 2024-03-04T17:50:42.356 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_vals_u64_empty_default (7 ms) 2024-03-04T17:50:42.356 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_value 2024-03-04T17:50:42.362 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_value (6 ms) 2024-03-04T17:50:42.362 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_vals_at_max_keys 2024-03-04T17:50:42.364 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_vals_at_max_keys (3 ms) 2024-03-04T17:50:42.364 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_vals_over_max_keys 2024-03-04T17:50:42.366 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_vals_over_max_keys (1 ms) 2024-03-04T17:50:42.366 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str 2024-03-04T17:50:42.377 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str (12 ms) 2024-03-04T17:50:42.377 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str_default 2024-03-04T17:50:42.395 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str_default (18 ms) 2024-03-04T17:50:42.395 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64 2024-03-04T17:50:42.408 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64 (13 ms) 2024-03-04T17:50:42.408 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64_default 2024-03-04T17:50:42.425 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64_default (17 ms) 2024-03-04T17:50:42.425 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_set_vals_str 2024-03-04T17:50:42.445 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_set_vals_str (19 ms) 2024-03-04T17:50:42.445 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64 2024-03-04T17:50:42.461 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_set_vals_u64 (16 ms) 2024-03-04T17:50:42.461 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_einval 2024-03-04T17:50:42.466 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_einval (6 ms) 2024-03-04T17:50:42.466 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_eio 2024-03-04T17:50:42.472 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_eio (6 ms) 2024-03-04T17:50:42.472 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_set_vals_at_max_keys 2024-03-04T17:50:42.476 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_set_vals_at_max_keys (3 ms) 2024-03-04T17:50:42.476 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_set_vals_over_max_keys 2024-03-04T17:50:42.478 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_set_vals_over_max_keys (2 ms) 2024-03-04T17:50:42.478 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_str 2024-03-04T17:50:42.495 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_str (18 ms) 2024-03-04T17:50:42.495 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_u64 2024-03-04T17:50:42.510 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_u64 (15 ms) 2024-03-04T17:50:42.510 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_rm_keys_str 2024-03-04T17:50:42.530 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_rm_keys_str (19 ms) 2024-03-04T17:50:42.530 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64 2024-03-04T17:50:42.549 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64 (19 ms) 2024-03-04T17:50:42.549 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_einval 2024-03-04T17:50:42.555 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_einval (5 ms) 2024-03-04T17:50:42.555 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_eio 2024-03-04T17:50:42.563 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_eio (8 ms) 2024-03-04T17:50:42.563 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_rm_keys_at_max_keys 2024-03-04T17:50:42.566 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_rm_keys_at_max_keys (3 ms) 2024-03-04T17:50:42.566 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_rm_keys_over_max_keys 2024-03-04T17:50:42.568 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_rm_keys_over_max_keys (2 ms) 2024-03-04T17:50:42.568 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_empty 2024-03-04T17:50:42.593 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_empty (26 ms) 2024-03-04T17:50:42.593 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 31 tests from CmpOmap (305 ms total) 2024-03-04T17:50:42.593 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:50:42.593 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:50:43.256 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 31 tests from 1 test suite ran. (2886 ms total) 2024-03-04T17:50:43.256 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 31 tests. 2024-03-04T17:50:43.261 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:50:43.261 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:50:43.319 INFO:tasks.workunit:Running workunits matching cls/test_cls_2pc_queue.sh on client.0... 2024-03-04T17:50:43.321 INFO:tasks.workunit:Running workunit cls/test_cls_2pc_queue.sh... 2024-03-04T17:50:43.321 DEBUG:teuthology.orchestra.run.smithi137:workunit test cls/test_cls_2pc_queue.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh 2024-03-04T17:50:43.387 INFO:tasks.workunit.client.0.smithi137.stdout:Running main() from gmock_main.cc 2024-03-04T17:50:43.388 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 24 tests from 1 test suite. 2024-03-04T17:50:43.388 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:50:43.388 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 24 tests from TestCls2PCQueue 2024-03-04T17:50:43.388 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.GetCapacity 2024-03-04T17:50:46.323 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.GetCapacity (2935 ms) 2024-03-04T17:50:46.323 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.AsyncGetCapacity 2024-03-04T17:50:49.323 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.AsyncGetCapacity (2998 ms) 2024-03-04T17:50:49.323 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.Reserve 2024-03-04T17:50:52.386 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.Reserve (3066 ms) 2024-03-04T17:50:52.387 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.AsyncReserve 2024-03-04T17:50:55.393 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.AsyncReserve (3004 ms) 2024-03-04T17:50:55.393 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.Commit 2024-03-04T17:50:59.472 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.Commit (4081 ms) 2024-03-04T17:50:59.473 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.Stats 2024-03-04T17:51:03.514 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.Stats (4043 ms) 2024-03-04T17:51:03.515 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.UpgradeFromReef 2024-03-04T17:51:08.587 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.UpgradeFromReef (5073 ms) 2024-03-04T17:51:08.588 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.Abort 2024-03-04T17:51:11.639 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.Abort (3048 ms) 2024-03-04T17:51:11.639 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.ReserveError 2024-03-04T17:51:14.649 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.ReserveError (3014 ms) 2024-03-04T17:51:14.650 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.CommitError 2024-03-04T17:51:17.695 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.CommitError (3044 ms) 2024-03-04T17:51:17.696 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.AbortError 2024-03-04T17:51:20.751 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.AbortError (3057 ms) 2024-03-04T17:51:20.751 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.MultiReserve 2024-03-04T17:51:23.779 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.MultiReserve (3022 ms) 2024-03-04T17:51:23.779 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.MultiCommit 2024-03-04T17:51:26.855 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.MultiCommit (3080 ms) 2024-03-04T17:51:26.855 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.MultiAbort 2024-03-04T17:51:29.877 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.MultiAbort (3025 ms) 2024-03-04T17:51:29.877 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.ReserveCommit 2024-03-04T17:51:32.889 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.ReserveCommit (3012 ms) 2024-03-04T17:51:32.889 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.ReserveAbort 2024-03-04T17:51:35.958 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.ReserveAbort (3067 ms) 2024-03-04T17:51:35.958 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.ManualCleanup 2024-03-04T17:51:38.965 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.ManualCleanup (3008 ms) 2024-03-04T17:51:38.965 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.Cleanup 2024-03-04T17:51:42.012 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.Cleanup (3048 ms) 2024-03-04T17:51:42.013 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.MultiProducer 2024-03-04T17:51:56.536 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.MultiProducer (14523 ms) 2024-03-04T17:51:56.536 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.AsyncConsumer 2024-03-04T17:52:00.685 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.AsyncConsumer (4150 ms) 2024-03-04T17:52:00.685 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.MultiProducerConsumer 2024-03-04T17:52:12.691 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.MultiProducerConsumer (12006 ms) 2024-03-04T17:52:12.691 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.ReserveSpillover 2024-03-04T17:52:19.859 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.ReserveSpillover (7168 ms) 2024-03-04T17:52:19.859 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.CommitSpillover 2024-03-04T17:52:28.315 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.CommitSpillover (8455 ms) 2024-03-04T17:52:28.315 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestCls2PCQueue.AbortSpillover 2024-03-04T17:52:36.304 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestCls2PCQueue.AbortSpillover (7990 ms) 2024-03-04T17:52:36.304 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 24 tests from TestCls2PCQueue (112917 ms total) 2024-03-04T17:52:36.304 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:36.304 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:36.304 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 24 tests from 1 test suite ran. (112917 ms total) 2024-03-04T17:52:36.304 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 24 tests. 2024-03-04T17:52:36.313 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:52:36.313 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:52:36.368 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2024-03-04T17:52:36.369 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2024-03-04T17:52:36.369 DEBUG:teuthology.orchestra.run.smithi137:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2024-03-04T17:52:36.445 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:36.445 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:36.446 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 6 tests from 1 test suite. 2024-03-04T17:52:36.446 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:38.315 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 6 tests from rgw_gc_log 2024-03-04T17:52:38.315 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2024-03-04T17:52:38.346 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] rgw_gc_log.init_existing_queue (31 ms) 2024-03-04T17:52:38.346 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2024-03-04T17:52:38.352 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] rgw_gc_log.init_existing_omap (6 ms) 2024-03-04T17:52:38.352 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2024-03-04T17:52:38.357 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (5 ms) 2024-03-04T17:52:38.357 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2024-03-04T17:52:38.360 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (2 ms) 2024-03-04T17:52:38.360 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2024-03-04T17:52:38.365 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] rgw_gc_log.defer1_after_init (5 ms) 2024-03-04T17:52:38.365 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2024-03-04T17:52:38.368 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] rgw_gc_log.defer2_before_init (3 ms) 2024-03-04T17:52:38.368 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 6 tests from rgw_gc_log (53 ms total) 2024-03-04T17:52:38.368 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:38.368 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:39.320 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 6 tests from 1 test suite ran. (2874 ms total) 2024-03-04T17:52:39.320 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 6 tests. 2024-03-04T17:52:39.328 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:52:39.328 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:52:39.383 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_obj.sh on client.0... 2024-03-04T17:52:39.384 INFO:tasks.workunit:Running workunit rgw/test_rgw_obj.sh... 2024-03-04T17:52:39.384 DEBUG:teuthology.orchestra.run.smithi137:workunit test rgw/test_rgw_obj.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_obj.sh 2024-03-04T17:52:39.461 INFO:tasks.workunit.client.0.smithi137.stdout:Running main() from gmock_main.cc 2024-03-04T17:52:39.461 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 4 tests from 1 test suite. 2024-03-04T17:52:39.461 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:39.461 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 4 tests from TestRGWObj 2024-03-04T17:52:39.461 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestRGWObj.underscore 2024-03-04T17:52:39.461 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj 2024-03-04T17:52:39.461 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj 2024-03-04T17:52:39.461 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: _obj ns= i= 2024-03-04T17:52:39.461 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj 2024-03-04T17:52:39.462 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj 2024-03-04T17:52:39.462 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: _obj ns= i= 2024-03-04T17:52:39.462 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj 2024-03-04T17:52:39.462 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj 2024-03-04T17:52:39.462 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: _obj ns= i= 2024-03-04T17:52:39.462 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.462 INFO:tasks.workunit.client.0.smithi137.stdout: "obj1": { 2024-03-04T17:52:39.462 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.462 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.462 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.462 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.462 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.462 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.462 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: "obj2": { 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.463 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.464 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.464 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.464 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.464 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.464 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.464 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.464 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.464 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.464 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.464 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.464 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.464 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.464 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.464 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3": { 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.465 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3-2": { 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.466 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.467 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.467 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.467 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.467 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.467 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.467 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.467 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj 2024-03-04T17:52:39.467 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj 2024-03-04T17:52:39.467 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: _obj ns=ns i= 2024-03-04T17:52:39.467 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj 2024-03-04T17:52:39.467 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj 2024-03-04T17:52:39.467 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: _obj ns=ns i= 2024-03-04T17:52:39.467 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj 2024-03-04T17:52:39.467 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj 2024-03-04T17:52:39.468 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: _obj ns=ns i= 2024-03-04T17:52:39.468 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.468 INFO:tasks.workunit.client.0.smithi137.stdout: "obj1": { 2024-03-04T17:52:39.468 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.468 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.468 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.468 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.468 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.468 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.468 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.468 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.468 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.468 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.468 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: "obj2": { 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.469 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.470 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.470 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.470 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.470 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.470 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.470 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.470 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.470 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.470 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.470 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3": { 2024-03-04T17:52:39.470 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.470 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.470 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.470 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3-2": { 2024-03-04T17:52:39.471 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.472 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.472 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.472 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.472 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.472 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.472 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.472 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.472 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.472 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.472 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.472 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.472 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.472 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.472 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj[v1] 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj[v1] 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: _obj ns= i=v1 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj[v1] 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj[v1] 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: _obj ns= i=v1 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj[v1] 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj[v1] 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: _obj ns= i=v1 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout: "obj1": { 2024-03-04T17:52:39.473 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "v1", 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.474 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.475 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.475 INFO:tasks.workunit.client.0.smithi137.stdout: "obj2": { 2024-03-04T17:52:39.475 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.475 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.475 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.475 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.475 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.475 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.475 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.475 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.475 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.475 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.475 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.475 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "v1", 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3": { 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.476 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3-2": { 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.477 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "v1", 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj[v1] 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj[v1] 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: _obj ns=ns i=v1 2024-03-04T17:52:39.478 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj[v1] 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj[v1] 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: _obj ns=ns i=v1 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj[v1] 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: _obj[v1] 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: _obj ns=ns i=v1 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout: "obj1": { 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.479 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "v1", 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: "obj2": { 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.480 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "v1", 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3": { 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.481 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.482 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3-2": { 2024-03-04T17:52:39.483 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.483 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.483 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.483 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.483 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.483 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.483 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.483 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.483 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.483 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.483 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.483 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.483 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_obj", 2024-03-04T17:52:39.484 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "v1", 2024-03-04T17:52:39.484 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.484 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.484 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.484 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.484 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestRGWObj.underscore (0 ms) 2024-03-04T17:52:39.484 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestRGWObj.no_underscore 2024-03-04T17:52:39.484 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj 2024-03-04T17:52:39.484 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj 2024-03-04T17:52:39.484 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: obj ns= i= 2024-03-04T17:52:39.484 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj 2024-03-04T17:52:39.484 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj 2024-03-04T17:52:39.484 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: obj ns= i= 2024-03-04T17:52:39.484 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: obj ns= i= 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout: "obj1": { 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.485 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.486 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.486 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.486 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.486 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.486 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.486 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.486 INFO:tasks.workunit.client.0.smithi137.stdout: "obj2": { 2024-03-04T17:52:39.486 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.486 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.486 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.486 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.486 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.486 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.486 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.487 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.487 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.487 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.487 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.487 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.487 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.487 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.487 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.487 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.487 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.487 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3": { 2024-03-04T17:52:39.487 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.487 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.487 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.488 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3-2": { 2024-03-04T17:52:39.489 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.489 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.489 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.489 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.489 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.489 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.489 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.489 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.489 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.489 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.489 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.489 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.489 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.489 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: obj ns=ns i= 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: obj ns=ns i= 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: obj ns=ns i= 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.490 INFO:tasks.workunit.client.0.smithi137.stdout: "obj1": { 2024-03-04T17:52:39.491 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.491 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.491 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.491 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.491 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.491 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.491 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.491 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.491 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.491 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.491 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.491 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.491 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.491 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.492 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.492 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.492 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.492 INFO:tasks.workunit.client.0.smithi137.stdout: "obj2": { 2024-03-04T17:52:39.492 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.492 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.492 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.492 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.492 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.492 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.492 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.492 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.492 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.492 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.493 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.493 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.493 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.493 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.493 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.493 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.493 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.493 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3": { 2024-03-04T17:52:39.493 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.493 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.493 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.493 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.493 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3-2": { 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.494 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.495 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.496 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.496 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj[v1] 2024-03-04T17:52:39.496 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj[v1] 2024-03-04T17:52:39.496 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: obj ns= i=v1 2024-03-04T17:52:39.496 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj[v1] 2024-03-04T17:52:39.496 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj[v1] 2024-03-04T17:52:39.496 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: obj ns= i=v1 2024-03-04T17:52:39.496 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj[v1] 2024-03-04T17:52:39.496 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj[v1] 2024-03-04T17:52:39.496 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: obj ns= i=v1 2024-03-04T17:52:39.496 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.496 INFO:tasks.workunit.client.0.smithi137.stdout: "obj1": { 2024-03-04T17:52:39.496 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.496 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "v1", 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.497 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: "obj2": { 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.498 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "v1", 2024-03-04T17:52:39.499 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.499 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.499 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.499 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3": { 2024-03-04T17:52:39.499 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.499 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.499 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.499 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.499 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.499 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.499 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.499 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.499 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.499 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3-2": { 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.500 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "v1", 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj[v1] 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj[v1] 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: obj ns=ns i=v1 2024-03-04T17:52:39.501 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj[v1] 2024-03-04T17:52:39.502 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj[v1] 2024-03-04T17:52:39.502 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: obj ns=ns i=v1 2024-03-04T17:52:39.502 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj[v1] 2024-03-04T17:52:39.502 INFO:tasks.workunit.client.0.smithi137.stdout:parsed: obj[v1] 2024-03-04T17:52:39.502 INFO:tasks.workunit.client.0.smithi137.stdout:stripped: obj ns=ns i=v1 2024-03-04T17:52:39.502 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.502 INFO:tasks.workunit.client.0.smithi137.stdout: "obj1": { 2024-03-04T17:52:39.502 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.502 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.502 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.502 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.502 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.502 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.502 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "v1", 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: "obj2": { 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.503 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "v1", 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3": { 2024-03-04T17:52:39.504 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.507 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.507 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.507 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.507 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.507 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.507 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.507 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.507 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.507 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: "obj3-2": { 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.508 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.509 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.509 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.509 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.509 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.509 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.509 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "obj", 2024-03-04T17:52:39.509 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "v1", 2024-03-04T17:52:39.509 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.509 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.509 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.509 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.509 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestRGWObj.no_underscore (0 ms) 2024-03-04T17:52:39.509 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestRGWObj.obj_to_raw 2024-03-04T17:52:39.509 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.510 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.511 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.512 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.512 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.512 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.512 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": "default-placement.data", 2024-03-04T17:52:39.512 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker_myobj", 2024-03-04T17:52:39.512 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.512 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.512 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.512 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.512 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.512 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.512 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.512 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.512 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.513 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.514 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.515 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.515 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.515 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.515 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.515 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.515 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.515 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.515 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.515 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.515 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.515 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.515 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.515 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.515 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.516 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.516 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.516 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.516 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.516 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.516 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.516 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker_myobj", 2024-03-04T17:52:39.516 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.516 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.516 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.516 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.516 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.516 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.517 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.518 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.519 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.519 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.519 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.519 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.519 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.519 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.519 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.519 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.519 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.519 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.519 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.519 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.519 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.519 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.520 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.520 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.520 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.520 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.520 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.520 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.520 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.520 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.520 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": "default-placement.data", 2024-03-04T17:52:39.520 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns_myobj", 2024-03-04T17:52:39.520 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.520 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.520 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.520 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.521 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.522 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.522 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.522 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.522 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.522 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.522 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.522 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.522 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.522 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.522 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.522 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.522 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.522 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.522 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.523 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.523 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.523 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.523 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.523 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.523 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.523 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.523 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.523 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.523 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.523 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.523 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.523 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.523 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.524 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.524 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.524 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.524 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.524 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.524 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.524 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.524 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.524 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.524 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.524 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.524 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.524 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns_myobj", 2024-03-04T17:52:39.524 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.525 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.525 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.525 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.525 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.525 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.525 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.525 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.525 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.525 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.525 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.525 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.525 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.525 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.525 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.526 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.526 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.526 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.526 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.526 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.526 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.526 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.526 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.526 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.526 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.526 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.526 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.526 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.526 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.527 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.528 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.528 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.528 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.528 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.528 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.528 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.528 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.528 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.528 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.528 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.528 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.528 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.528 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.528 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.529 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": "default-placement.data", 2024-03-04T17:52:39.529 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__:inst_myobj", 2024-03-04T17:52:39.529 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.529 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.529 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.529 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.529 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.529 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.529 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.529 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.529 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.529 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.529 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.529 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.530 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.531 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.531 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.531 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.531 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.531 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.531 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.531 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.531 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.531 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.531 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.531 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.531 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.531 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.532 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.532 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.532 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.532 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.532 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.532 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.532 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.532 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.532 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.532 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.532 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.532 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.532 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.532 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__:inst_myobj", 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.533 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.534 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.534 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.534 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.534 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.534 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.534 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.534 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.534 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.534 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.534 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.534 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.534 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.534 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.534 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.535 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.536 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.536 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.536 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.536 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.536 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.536 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.536 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.536 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.536 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.536 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.536 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.536 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.536 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.536 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.537 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.537 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.537 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.537 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.537 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": "default-placement.data", 2024-03-04T17:52:39.537 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns:inst_myobj", 2024-03-04T17:52:39.537 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.537 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.537 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.537 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.537 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.537 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.537 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.537 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.538 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.538 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.538 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.538 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.538 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.538 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.538 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.538 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.538 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.538 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.538 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.538 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.538 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.538 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.539 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.540 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.541 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.541 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.541 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.541 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.541 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.541 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.541 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.541 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns:inst_myobj", 2024-03-04T17:52:39.541 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.541 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.541 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.541 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.541 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.541 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.542 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.543 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.543 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.543 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.543 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.543 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.543 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.543 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.543 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.543 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.543 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.543 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.543 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.543 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.543 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.544 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.544 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.544 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.544 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.544 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.544 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.544 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.544 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.544 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.544 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.544 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.544 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.544 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.545 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.545 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.545 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.545 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.545 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.545 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.545 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.545 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.545 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.545 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.545 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": "default-placement.data", 2024-03-04T17:52:39.545 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker___myobj", 2024-03-04T17:52:39.545 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj" 2024-03-04T17:52:39.545 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.546 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.546 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.546 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.546 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.546 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.546 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.546 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.546 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.546 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.546 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.546 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.546 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.546 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.546 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.547 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.547 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.547 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.547 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.547 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.547 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.547 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.547 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.547 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.547 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.547 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.547 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.547 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.547 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.548 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.548 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.548 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.548 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.548 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.548 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.548 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.548 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.548 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.548 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.548 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.548 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.548 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.548 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.549 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.549 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.549 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.549 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.549 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.549 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.549 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.549 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.549 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.549 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.549 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.549 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.549 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.549 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.550 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.550 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker___myobj", 2024-03-04T17:52:39.550 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj" 2024-03-04T17:52:39.550 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.550 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.550 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.550 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.550 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.550 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.550 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.550 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.550 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.550 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.550 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.551 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.551 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.551 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.551 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.551 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.551 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.551 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.551 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.551 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.551 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.551 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.551 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.551 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.551 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.552 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.552 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.552 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.552 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.552 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.552 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.552 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.552 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.552 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.552 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.552 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.552 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.552 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.552 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.553 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.554 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.554 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.554 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.554 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": "default-placement.data", 2024-03-04T17:52:39.554 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns__myobj", 2024-03-04T17:52:39.554 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.554 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.554 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.554 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.554 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.554 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.554 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.554 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.555 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.556 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.557 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.557 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.557 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.557 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.557 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.557 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.557 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.557 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.557 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.557 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.557 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.557 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.557 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.557 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns__myobj", 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.558 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.559 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.560 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.560 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.560 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.560 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.560 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.560 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.560 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.560 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.560 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.560 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.560 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.560 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.560 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.561 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.561 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.561 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.561 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.561 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.561 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.561 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.561 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.561 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.561 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.561 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.561 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.561 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.561 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": "default-placement.data", 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__:inst__myobj", 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj" 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.562 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.563 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.563 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.563 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.563 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.563 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.563 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.563 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.563 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.563 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.563 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.563 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.563 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.563 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.563 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.564 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.564 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.564 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.564 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.564 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.564 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.564 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.564 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.564 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.564 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.564 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.564 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.564 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.564 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.565 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.565 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.565 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.565 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.565 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.565 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.565 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.565 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.565 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.565 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.565 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.565 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.565 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.565 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__:inst__myobj", 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj" 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.566 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.567 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.567 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.567 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.567 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.567 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.567 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.567 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.567 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.567 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.567 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.567 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.567 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.567 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.567 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.568 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.569 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.569 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.569 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.569 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.569 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.569 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.569 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.569 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.569 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.569 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.569 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.569 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.569 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.569 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.570 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.570 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.570 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.570 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.570 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.570 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.570 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.570 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.570 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.570 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.570 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.570 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": "default-placement.data", 2024-03-04T17:52:39.570 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns:inst__myobj", 2024-03-04T17:52:39.570 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.571 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.572 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.573 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns:inst__myobj", 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.574 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.575 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.576 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.577 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": "default-placement.data", 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker___myobj_", 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.578 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.579 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.580 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.581 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker___myobj_", 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.582 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.583 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.584 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.584 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.584 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.584 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.584 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.584 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.584 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.584 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.584 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.584 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.584 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.584 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.584 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.584 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.585 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": "default-placement.data", 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns__myobj_", 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.586 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.587 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.588 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.588 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.588 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.588 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.588 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.588 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.588 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.588 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.588 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.588 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.588 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.588 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.588 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.588 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.589 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.589 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.589 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.589 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.589 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.589 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.589 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.589 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.589 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.589 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.589 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.589 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.589 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.589 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.590 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns__myobj_", 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.591 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.592 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.592 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.592 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.592 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.592 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.592 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.592 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.592 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.592 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.592 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.592 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.592 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.592 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.592 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.593 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.594 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.594 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.594 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.594 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.594 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.594 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.594 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.594 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.594 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.594 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.594 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.594 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.594 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.594 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": "default-placement.data", 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__:inst__myobj_", 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.595 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.596 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.596 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.596 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.596 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.596 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.596 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.596 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.596 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.596 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.596 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.596 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.596 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.596 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.596 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.597 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.598 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.598 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.598 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.598 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.598 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.598 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.598 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.598 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.598 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.598 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.598 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.598 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.598 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.599 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.599 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.599 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.599 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.599 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__:inst__myobj_", 2024-03-04T17:52:39.599 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:39.599 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.599 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.599 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.599 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.599 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.599 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.599 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.599 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.600 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.600 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.600 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.600 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.600 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.600 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.600 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.600 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.600 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.600 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.600 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.600 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.600 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.600 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.601 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.602 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.602 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.602 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.602 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.602 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.602 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.602 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.602 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.602 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.602 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.602 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.602 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.602 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.602 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": "default-placement.data", 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns:inst__myobj_", 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.603 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "test", 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": "", 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": "" 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.604 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.605 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.605 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.605 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.605 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.605 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.605 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.605 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.605 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.605 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.605 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.605 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.605 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.605 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.605 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout: "obj": { 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.606 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns:inst__myobj_", 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.607 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.608 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestRGWObj.obj_to_raw (0 ms) 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] TestRGWObj.old_to_raw 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker_myobj", 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.609 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.610 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout:raw=.data-pool:marker_myobj 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj2": { 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker_myobj", 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj2": { 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.611 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.612 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.612 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.612 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.612 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.612 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.612 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.612 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.612 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.612 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.612 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.612 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.612 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.612 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.612 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout:raw2=.data-pool:marker_myobj 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns_myobj", 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.613 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.614 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.614 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.614 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.614 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.614 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.614 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.614 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.614 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.614 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.614 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.614 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.614 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.614 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.614 INFO:tasks.workunit.client.0.smithi137.stdout:raw=.data-pool:marker__ns_myobj 2024-03-04T17:52:39.615 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.615 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj2": { 2024-03-04T17:52:39.615 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.615 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns_myobj", 2024-03-04T17:52:39.615 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.615 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.615 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.615 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.615 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj2": { 2024-03-04T17:52:39.615 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.615 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.615 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.615 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.615 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout:raw2=.data-pool:marker__ns_myobj 2024-03-04T17:52:39.616 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__:inst_myobj", 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.617 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout:raw=.data-pool:marker__:inst_myobj 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj2": { 2024-03-04T17:52:39.618 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__:inst_myobj", 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj2": { 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.619 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout:raw2=.data-pool:marker__:inst_myobj 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns:inst_myobj", 2024-03-04T17:52:39.620 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.621 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.622 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.622 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.622 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.622 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.622 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.622 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.622 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.622 INFO:tasks.workunit.client.0.smithi137.stdout:raw=.data-pool:marker__ns:inst_myobj 2024-03-04T17:52:39.622 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.622 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj2": { 2024-03-04T17:52:39.622 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.622 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns:inst_myobj", 2024-03-04T17:52:39.622 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.622 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj2": { 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.623 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "myobj", 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout:raw2=.data-pool:marker__ns:inst_myobj 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker___myobj", 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj" 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.624 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.625 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.625 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.625 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.625 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.625 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.625 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.625 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.625 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.625 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.625 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.625 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.625 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.625 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.625 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout:raw=.data-pool:marker___myobj 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj2": { 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker___myobj", 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj" 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.626 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj2": { 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.627 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout:raw2=.data-pool:marker___myobj 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns__myobj", 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.628 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.629 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout:raw=.data-pool:marker__ns__myobj 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj2": { 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns__myobj", 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj2": { 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.630 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.631 INFO:tasks.workunit.client.0.smithi137.stdout:raw2=.data-pool:marker__ns__myobj 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__:inst__myobj", 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj" 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.632 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout:raw=.data-pool:marker__:inst__myobj 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj2": { 2024-03-04T17:52:39.633 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__:inst__myobj", 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj" 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj2": { 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.634 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout:raw2=.data-pool:marker__:inst__myobj 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns:inst__myobj", 2024-03-04T17:52:39.635 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.636 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.636 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.636 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.636 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.636 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.636 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.636 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.636 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.636 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.636 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.636 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.636 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.636 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.636 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout:raw=.data-pool:marker__ns:inst__myobj 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj2": { 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns:inst__myobj", 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.637 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj2": { 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.638 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj", 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout:raw2=.data-pool:marker__ns:inst__myobj 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker___myobj_", 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.639 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.640 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.641 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.641 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.641 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.641 INFO:tasks.workunit.client.0.smithi137.stdout:raw=.data-pool:marker___myobj_ 2024-03-04T17:52:39.641 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.641 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj2": { 2024-03-04T17:52:39.641 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.641 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker___myobj_", 2024-03-04T17:52:39.641 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:39.641 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.641 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.641 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.641 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj2": { 2024-03-04T17:52:39.641 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.642 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout:raw2=.data-pool:marker___myobj_ 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns__myobj_", 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.643 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.644 INFO:tasks.workunit.client.0.smithi137.stdout:raw=.data-pool:marker__ns__myobj_ 2024-03-04T17:52:39.645 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.645 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj2": { 2024-03-04T17:52:39.645 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.645 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns__myobj_", 2024-03-04T17:52:39.645 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.645 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.645 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.645 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.645 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj2": { 2024-03-04T17:52:39.645 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.645 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.645 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.645 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.645 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "", 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout:raw2=.data-pool:marker__ns__myobj_ 2024-03-04T17:52:39.646 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__:inst__myobj_", 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.647 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout:raw=.data-pool:marker__:inst__myobj_ 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj2": { 2024-03-04T17:52:39.648 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__:inst__myobj_", 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "marker__myobj_" 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj2": { 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.649 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "" 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout:raw2=.data-pool:marker__:inst__myobj_ 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj": { 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns:inst__myobj_", 2024-03-04T17:52:39.650 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj": { 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.651 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.652 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.652 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.652 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.652 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.652 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.652 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.652 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.652 INFO:tasks.workunit.client.0.smithi137.stdout:raw=.data-pool:marker__ns:inst__myobj_ 2024-03-04T17:52:39.652 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.652 INFO:tasks.workunit.client.0.smithi137.stdout: "raw_obj2": { 2024-03-04T17:52:39.652 INFO:tasks.workunit.client.0.smithi137.stdout: "pool": ".data-pool", 2024-03-04T17:52:39.652 INFO:tasks.workunit.client.0.smithi137.stdout: "oid": "marker__ns:inst__myobj_", 2024-03-04T17:52:39.652 INFO:tasks.workunit.client.0.smithi137.stdout: "loc": "" 2024-03-04T17:52:39.652 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout: "new_obj2": { 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket": { 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "ebtest", 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout: "marker": "marker", 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_id": "bucket-id", 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout: "tenant": "", 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout: "explicit_placement": { 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout: "data_pool": ".data-pool", 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout: "data_extra_pool": "", 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout: "index_pool": ".index-pool" 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:39.653 INFO:tasks.workunit.client.0.smithi137.stdout: "key": { 2024-03-04T17:52:39.654 INFO:tasks.workunit.client.0.smithi137.stdout: "name": "_myobj_", 2024-03-04T17:52:39.654 INFO:tasks.workunit.client.0.smithi137.stdout: "instance": "inst", 2024-03-04T17:52:39.654 INFO:tasks.workunit.client.0.smithi137.stdout: "ns": "ns" 2024-03-04T17:52:39.654 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.654 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:39.654 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:39.654 INFO:tasks.workunit.client.0.smithi137.stdout:raw2=.data-pool:marker__ns:inst__myobj_ 2024-03-04T17:52:39.654 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] TestRGWObj.old_to_raw (1 ms) 2024-03-04T17:52:39.654 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 4 tests from TestRGWObj (1 ms total) 2024-03-04T17:52:39.654 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:39.654 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:39.654 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 4 tests from 1 test suite ran. (1 ms total) 2024-03-04T17:52:39.654 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 4 tests. 2024-03-04T17:52:39.655 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:52:39.655 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:52:39.669 INFO:tasks.workunit:Running workunits matching rgw/test_librgw_file.sh on client.0... 2024-03-04T17:52:39.670 INFO:tasks.workunit:Running workunit rgw/test_librgw_file.sh... 2024-03-04T17:52:39.671 DEBUG:teuthology.orchestra.run.smithi137:workunit test rgw/test_librgw_file.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_librgw_file.sh 2024-03-04T17:52:39.767 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:39.767 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:39.810 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:39.810+0000 7f62de6e1980 20 rados->read ofs=0 len=0 2024-03-04T17:52:39.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:39.814+0000 7f62de6e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:39.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:39.814+0000 7f62de6e1980 20 realm 2024-03-04T17:52:39.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:39.814+0000 7f62de6e1980 20 rados->read ofs=0 len=0 2024-03-04T17:52:39.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:39.814+0000 7f62de6e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:39.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:39.814+0000 7f62de6e1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:52:39.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:39.814+0000 7f62de6e1980 20 rados->read ofs=0 len=0 2024-03-04T17:52:39.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:39.814+0000 7f62de6e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:39.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:39.814+0000 7f62de6e1980 20 rados->read ofs=0 len=0 2024-03-04T17:52:39.814 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:39.814+0000 7f62de6e1980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:52:39.814 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:39.814+0000 7f62de6e1980 20 rados->read ofs=0 len=0 2024-03-04T17:52:39.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:39.818+0000 7f62de6e1980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:52:39.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:39.818+0000 7f62de6e1980 20 searching for the correct realm 2024-03-04T17:52:39.820 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:39.822+0000 7f62dc9fa640 0 -- 172.21.15.137:0/1587689598 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f62d40faca0 msgr2=0x7f62d41069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:52:40.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.038+0000 7f62dc9fa640 0 -- 172.21.15.137:0/1587689598 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x5625572e4ce0 msgr2=0x5625572e71a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:52:40.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.046+0000 7f62de6e1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:52:40.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.046+0000 7f62de6e1980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:52:40.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.046+0000 7f62de6e1980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:52:40.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.046+0000 7f62de6e1980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:52:40.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.046+0000 7f62de6e1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:52:40.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.046+0000 7f62de6e1980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:52:40.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.046+0000 7f62de6e1980 20 rados->read ofs=0 len=0 2024-03-04T17:52:40.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:40.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 rados->read ofs=0 len=0 2024-03-04T17:52:40.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:52:40.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 rados->read ofs=0 len=0 2024-03-04T17:52:40.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:52:40.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 zone default found 2024-03-04T17:52:40.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 4 Realm: () 2024-03-04T17:52:40.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:52:40.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:52:40.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:52:40.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 zonegroup default 2024-03-04T17:52:40.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 rados->read ofs=0 len=0 2024-03-04T17:52:40.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:40.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 rados->read ofs=0 len=0 2024-03-04T17:52:40.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:40.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 rados->read ofs=0 len=0 2024-03-04T17:52:40.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:40.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 started sync module instance, tier type = 2024-03-04T17:52:40.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.050+0000 7f62de6e1980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:52:40.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.074+0000 7f62de6e1980 20 add_watcher() i=0 2024-03-04T17:52:40.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.074+0000 7f62de6e1980 20 add_watcher() i=1 2024-03-04T17:52:40.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.074+0000 7f62de6e1980 20 add_watcher() i=2 2024-03-04T17:52:40.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.074+0000 7f62de6e1980 20 add_watcher() i=3 2024-03-04T17:52:40.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.074+0000 7f62de6e1980 20 add_watcher() i=4 2024-03-04T17:52:40.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.074+0000 7f62de6e1980 20 add_watcher() i=5 2024-03-04T17:52:40.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.074+0000 7f62de6e1980 20 add_watcher() i=6 2024-03-04T17:52:40.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.078+0000 7f62de6e1980 20 add_watcher() i=7 2024-03-04T17:52:40.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.078+0000 7f62de6e1980 2 all 8 watchers are set, enabling cache 2024-03-04T17:52:40.079 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f6192fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:52:40.080 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:52:40.080 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:52:40.080 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 5 note: GC not initialized 2024-03-04T17:52:40.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f6191fdb640 20 reqs_thread_entry: start 2024-03-04T17:52:40.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 20 init_complete bucket index max shards: 1 2024-03-04T17:52:40.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 20 Filter name: none 2024-03-04T17:52:40.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f61037fe640 20 reqs_thread_entry: start 2024-03-04T17:52:40.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 10 cache get: name=default.rgw.meta+users.uid+ceph-test-librgw-file : miss 2024-03-04T17:52:40.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 20 rados->read ofs=0 len=0 2024-03-04T17:52:40.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:40.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x0 2024-03-04T17:52:40.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 10 adding default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-03-04T17:52:40.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 10 cache get: name=default.rgw.meta+users.email+librgw@example.com : miss 2024-03-04T17:52:40.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 20 rados->read ofs=0 len=0 2024-03-04T17:52:40.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:40.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x0 2024-03-04T17:52:40.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 10 adding default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-03-04T17:52:40.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 10 cache get: name=default.rgw.meta+users.keys+wHX4/LLc+DZqBScGQmIStE7ZBfg= : miss 2024-03-04T17:52:40.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.082+0000 7f62de6e1980 20 rados->read ofs=0 len=0 2024-03-04T17:52:40.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.086+0000 7f62de6e1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:40.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.086+0000 7f62de6e1980 10 cache put: name=default.rgw.meta+users.keys+wHX4/LLc+DZqBScGQmIStE7ZBfg= info.flags=0x0 2024-03-04T17:52:40.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.086+0000 7f62de6e1980 10 adding default.rgw.meta+users.keys+wHX4/LLc+DZqBScGQmIStE7ZBfg= to cache LRU end 2024-03-04T17:52:40.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.086+0000 7f62de6e1980 10 cache get: name=default.rgw.meta+users.keys+wHX4/LLc+DZqBScGQmIStE7ZBfg= : hit (negative entry) 2024-03-04T17:52:40.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.086+0000 7f62de6e1980 10 cache get: name=default.rgw.meta+users.keys+wHX4/LLc+DZqBScGQmIStE7ZBfg= : hit (negative entry) 2024-03-04T17:52:40.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.086+0000 7f62de6e1980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2024-03-04T17:52:40.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.086+0000 7f62de6e1980 10 moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-03-04T17:52:40.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.086+0000 7f62de6e1980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-librgw-file, ofs0, ns] 2024-03-04T17:52:40.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.086+0000 7f61b37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1060856922117 cookie 94718376416784 notifier 5487 bl.length()=664 2024-03-04T17:52:40.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.086+0000 7f61b37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2024-03-04T17:52:40.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.086+0000 7f61b37fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-03-04T17:52:40.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.090+0000 7f62de6e1980 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2024-03-04T17:52:40.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.090+0000 7f62de6e1980 10 moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-03-04T17:52:40.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.090+0000 7f62de6e1980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:librgw@example.com, ofs0, ns] 2024-03-04T17:52:40.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.090+0000 7f61b37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1060856922112 cookie 94718376433456 notifier 5487 bl.length()=194 2024-03-04T17:52:40.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.090+0000 7f61b37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2024-03-04T17:52:40.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.090+0000 7f61b37fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-03-04T17:52:40.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.094+0000 7f62de6e1980 10 cache put: name=default.rgw.meta+users.keys+wHX4/LLc+DZqBScGQmIStE7ZBfg= info.flags=0x7 2024-03-04T17:52:40.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.094+0000 7f62de6e1980 10 moving default.rgw.meta+users.keys+wHX4/LLc+DZqBScGQmIStE7ZBfg= to cache LRU end 2024-03-04T17:52:40.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.094+0000 7f62de6e1980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:wHX4/LLc+DZqBScGQmIStE7ZBfg=, ofs0, ns] 2024-03-04T17:52:40.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.094+0000 7f61b2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1060856922114 cookie 94718376435664 notifier 5487 bl.length()=203 2024-03-04T17:52:40.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.094+0000 7f61b2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+wHX4/LLc+DZqBScGQmIStE7ZBfg= info.flags=0x7 2024-03-04T17:52:40.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.094+0000 7f61b2ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+wHX4/LLc+DZqBScGQmIStE7ZBfg= to cache LRU end 2024-03-04T17:52:40.096 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:40.096 INFO:tasks.workunit.client.0.smithi137.stdout: "user_id": "ceph-test-librgw-file", 2024-03-04T17:52:40.096 INFO:tasks.workunit.client.0.smithi137.stdout: "display_name": "librgw test user", 2024-03-04T17:52:40.096 INFO:tasks.workunit.client.0.smithi137.stdout: "email": "librgw@example.com", 2024-03-04T17:52:40.096 INFO:tasks.workunit.client.0.smithi137.stdout: "suspended": 0, 2024-03-04T17:52:40.096 INFO:tasks.workunit.client.0.smithi137.stdout: "max_buckets": 1000, 2024-03-04T17:52:40.096 INFO:tasks.workunit.client.0.smithi137.stdout: "subusers": [], 2024-03-04T17:52:40.097 INFO:tasks.workunit.client.0.smithi137.stdout: "keys": [ 2024-03-04T17:52:40.097 INFO:tasks.workunit.client.0.smithi137.stdout: { 2024-03-04T17:52:40.097 INFO:tasks.workunit.client.0.smithi137.stdout: "user": "ceph-test-librgw-file", 2024-03-04T17:52:40.097 INFO:tasks.workunit.client.0.smithi137.stdout: "access_key": "wHX4/LLc+DZqBScGQmIStE7ZBfg=", 2024-03-04T17:52:40.097 INFO:tasks.workunit.client.0.smithi137.stdout: "secret_key": "TPJvAfDQh3rvdUO9Izss/bqnSREEv5EbxFppSnUGP2CFe32LSpLQZg==", 2024-03-04T17:52:40.097 INFO:tasks.workunit.client.0.smithi137.stdout: "active": true 2024-03-04T17:52:40.097 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:40.097 INFO:tasks.workunit.client.0.smithi137.stdout: ], 2024-03-04T17:52:40.097 INFO:tasks.workunit.client.0.smithi137.stdout: "swift_keys": [], 2024-03-04T17:52:40.097 INFO:tasks.workunit.client.0.smithi137.stdout: "caps": [], 2024-03-04T17:52:40.097 INFO:tasks.workunit.client.0.smithi137.stdout: "op_mask": "read, write, delete", 2024-03-04T17:52:40.097 INFO:tasks.workunit.client.0.smithi137.stdout: "default_placement": "", 2024-03-04T17:52:40.097 INFO:tasks.workunit.client.0.smithi137.stdout: "default_storage_class": "", 2024-03-04T17:52:40.097 INFO:tasks.workunit.client.0.smithi137.stdout: "placement_tags": [], 2024-03-04T17:52:40.098 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_quota": { 2024-03-04T17:52:40.098 INFO:tasks.workunit.client.0.smithi137.stdout: "enabled": false, 2024-03-04T17:52:40.098 INFO:tasks.workunit.client.0.smithi137.stdout: "check_on_raw": false, 2024-03-04T17:52:40.098 INFO:tasks.workunit.client.0.smithi137.stdout: "max_size": -1, 2024-03-04T17:52:40.098 INFO:tasks.workunit.client.0.smithi137.stdout: "max_size_kb": 0, 2024-03-04T17:52:40.098 INFO:tasks.workunit.client.0.smithi137.stdout: "max_objects": -1 2024-03-04T17:52:40.098 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:40.098 INFO:tasks.workunit.client.0.smithi137.stdout: "user_quota": { 2024-03-04T17:52:40.098 INFO:tasks.workunit.client.0.smithi137.stdout: "enabled": false, 2024-03-04T17:52:40.098 INFO:tasks.workunit.client.0.smithi137.stdout: "check_on_raw": false, 2024-03-04T17:52:40.098 INFO:tasks.workunit.client.0.smithi137.stdout: "max_size": -1, 2024-03-04T17:52:40.098 INFO:tasks.workunit.client.0.smithi137.stdout: "max_size_kb": 0, 2024-03-04T17:52:40.098 INFO:tasks.workunit.client.0.smithi137.stdout: "max_objects": -1 2024-03-04T17:52:40.098 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:40.099 INFO:tasks.workunit.client.0.smithi137.stdout: "temp_url_keys": [], 2024-03-04T17:52:40.099 INFO:tasks.workunit.client.0.smithi137.stdout: "type": "rgw", 2024-03-04T17:52:40.099 INFO:tasks.workunit.client.0.smithi137.stdout: "mfa_ids": [] 2024-03-04T17:52:40.099 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:40.099 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:40.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.098+0000 7f6192fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:52:40.102 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.102+0000 7f62de6e1980 20 remove_watcher() i=0 2024-03-04T17:52:40.102 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.102+0000 7f62de6e1980 2 removed watcher, disabling cache 2024-03-04T17:52:40.104 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.106+0000 7f62de6e1980 20 remove_watcher() i=1 2024-03-04T17:52:40.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.106+0000 7f62de6e1980 20 remove_watcher() i=2 2024-03-04T17:52:40.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.110+0000 7f62de6e1980 20 remove_watcher() i=3 2024-03-04T17:52:40.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.110+0000 7f62de6e1980 20 remove_watcher() i=4 2024-03-04T17:52:40.112 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.114+0000 7f62de6e1980 20 remove_watcher() i=5 2024-03-04T17:52:40.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.118+0000 7f62de6e1980 20 remove_watcher() i=6 2024-03-04T17:52:40.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:40.118+0000 7f62de6e1980 20 remove_watcher() i=7 2024-03-04T17:52:40.132 INFO:tasks.workunit.client.0.smithi137.stdout:phase 1.1 2024-03-04T17:52:40.153 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 24 tests from 1 test suite. 2024-03-04T17:52:40.153 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:40.153 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 24 tests from LibRGW 2024-03-04T17:52:40.153 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:52:40.166 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:40.166 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:40.762 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.INIT (609 ms) 2024-03-04T17:52:40.763 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-03-04T17:52:40.764 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MOUNT_NOROOT (2 ms) 2024-03-04T17:52:40.764 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:52:40.764 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-03-04T17:52:40.764 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-03-04T17:52:40.780 INFO:tasks.workunit.client.0.smithi137.stdout:creating: nfsroot:foo/bar/baz/quux 2024-03-04T17:52:40.789 INFO:tasks.workunit.client.0.smithi137.stdout:creating: nfsroot:foo/f1 2024-03-04T17:52:40.795 INFO:tasks.workunit.client.0.smithi137.stdout:creating: nfsroot:foo/f2 2024-03-04T17:52:40.800 INFO:tasks.workunit.client.0.smithi137.stdout:creating: nfsroot:foo/bar/f1 2024-03-04T17:52:40.804 INFO:tasks.workunit.client.0.smithi137.stdout:creating: nfsroot:foo/bar/d1/ 2024-03-04T17:52:40.810 INFO:tasks.workunit.client.0.smithi137.stdout:creating: nfsroot:foo/bar/baz/hungry 2024-03-04T17:52:40.816 INFO:tasks.workunit.client.0.smithi137.stdout:creating: nfsroot:foo/bar/baz/hungry/ 2024-03-04T17:52:40.820 INFO:tasks.workunit.client.0.smithi137.stdout:creating: nfsroot:foo/bar/baz/momma 2024-03-04T17:52:40.826 INFO:tasks.workunit.client.0.smithi137.stdout:creating: nfsroot:foo/bar/baz/bear/ 2024-03-04T17:52:40.832 INFO:tasks.workunit.client.0.smithi137.stdout:creating: nfsroot:foo/bar/baz/sasquatch 2024-03-04T17:52:40.836 INFO:tasks.workunit.client.0.smithi137.stdout:creating: nfsroot:foo/bar/baz/sasquatch/ 2024-03-04T17:52:40.842 INFO:tasks.workunit.client.0.smithi137.stdout:creating: nfsroot:foo/bar/baz/frobozz 2024-03-04T17:52:40.848 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SETUP_HIER1 (84 ms) 2024-03-04T17:52:40.848 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-03-04T17:52:41.007 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SETUP_DIRS1 (159 ms) 2024-03-04T17:52:41.008 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SETATTR 2024-03-04T17:52:41.008 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-03-04T17:52:41.008 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-03-04T17:52:41.037 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (30 ms) 2024-03-04T17:52:41.037 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-03-04T17:52:41.129 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (93 ms) 2024-03-04T17:52:41.130 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-03-04T17:52:41.130 INFO:tasks.workunit.client.0.smithi137.stdout:rename file rfile_0 to rfile_r0 (bucket brename0) 2024-03-04T17:52:41.145 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (15 ms) 2024-03-04T17:52:41.145 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-03-04T17:52:41.145 INFO:tasks.workunit.client.0.smithi137.stdout:rename file rfile_1 (bucket brename0) to rfile_rhilldog (bucket brename1) 2024-03-04T17:52:41.164 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (18 ms) 2024-03-04T17:52:41.164 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-03-04T17:52:41.164 INFO:tasks.workunit.client.0.smithi137.stdout:scanning objects in /bdirs1/dir_0 2024-03-04T17:52:41.165 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-03-04T17:52:41.165 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-03-04T17:52:41.165 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-03-04T17:52:41.165 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-03-04T17:52:41.165 INFO:tasks.workunit.client.0.smithi137.stdout:scanning objects in /bdirs1/dir_1 2024-03-04T17:52:41.165 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-03-04T17:52:41.165 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-03-04T17:52:41.165 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-03-04T17:52:41.165 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-03-04T17:52:41.165 INFO:tasks.workunit.client.0.smithi137.stdout:scanning objects in /bdirs1/dir_2 2024-03-04T17:52:41.165 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-03-04T17:52:41.165 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-03-04T17:52:41.165 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-03-04T17:52:41.165 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-03-04T17:52:41.166 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-03-04T17:52:41.166 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-03-04T17:52:41.166 INFO:tasks.workunit.client.0.smithi137.stdout:read back objects in /bdirs1/dir_0 2024-03-04T17:52:41.166 INFO:tasks.workunit.client.0.smithi137.stdout:reading 0,256 dir_0/sfile_0 2024-03-04T17:52:41.167 INFO:tasks.workunit.client.0.smithi137.stdout: read back from sfile_0 : "data for sfile_0" 2024-03-04T17:52:41.168 INFO:tasks.workunit.client.0.smithi137.stdout:reading 0,256 dir_0/sfile_1 2024-03-04T17:52:41.169 INFO:tasks.workunit.client.0.smithi137.stdout: read back from sfile_1 : "data for sfile_1" 2024-03-04T17:52:41.169 INFO:tasks.workunit.client.0.smithi137.stdout:read back objects in /bdirs1/dir_1 2024-03-04T17:52:41.170 INFO:tasks.workunit.client.0.smithi137.stdout:reading 0,256 dir_1/sfile_0 2024-03-04T17:52:41.171 INFO:tasks.workunit.client.0.smithi137.stdout: read back from sfile_0 : "data for sfile_0" 2024-03-04T17:52:41.171 INFO:tasks.workunit.client.0.smithi137.stdout:reading 0,256 dir_1/sfile_1 2024-03-04T17:52:41.174 INFO:tasks.workunit.client.0.smithi137.stdout: read back from sfile_1 : "data for sfile_1" 2024-03-04T17:52:41.174 INFO:tasks.workunit.client.0.smithi137.stdout:read back objects in /bdirs1/dir_2 2024-03-04T17:52:41.174 INFO:tasks.workunit.client.0.smithi137.stdout:reading 0,256 dir_2/sfile_0 2024-03-04T17:52:41.177 INFO:tasks.workunit.client.0.smithi137.stdout: read back from sfile_0 : "data for sfile_0" 2024-03-04T17:52:41.177 INFO:tasks.workunit.client.0.smithi137.stdout:reading 0,256 dir_2/sfile_1 2024-03-04T17:52:41.179 INFO:tasks.workunit.client.0.smithi137.stdout: read back from sfile_1 : "data for sfile_1" 2024-03-04T17:52:41.179 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.READ_DIRS1 (15 ms) 2024-03-04T17:52:41.179 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-03-04T17:52:41.408 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.READF_SETUP1 (229 ms) 2024-03-04T17:52:41.408 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-03-04T17:52:41.408 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-03-04T17:52:41.408 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-03-04T17:52:41.408 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-03-04T17:52:41.408 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-03-04T17:52:41.408 INFO:tasks.workunit.client.0.smithi137.stdout:release sdir_0 type: DIR refs: 2 2024-03-04T17:52:41.408 INFO:tasks.workunit.client.0.smithi137.stdout:release sfile_0 type: FILE refs: 2 2024-03-04T17:52:41.408 INFO:tasks.workunit.client.0.smithi137.stdout:release sdir_1 type: DIR refs: 2 2024-03-04T17:52:41.408 INFO:tasks.workunit.client.0.smithi137.stdout:release sfile_1 type: FILE refs: 2 2024-03-04T17:52:41.408 INFO:tasks.workunit.client.0.smithi137.stdout:release sdir_0 type: DIR refs: 2 2024-03-04T17:52:41.409 INFO:tasks.workunit.client.0.smithi137.stdout:release sfile_0 type: FILE refs: 2 2024-03-04T17:52:41.409 INFO:tasks.workunit.client.0.smithi137.stdout:release sdir_1 type: DIR refs: 2 2024-03-04T17:52:41.409 INFO:tasks.workunit.client.0.smithi137.stdout:release sfile_1 type: FILE refs: 2 2024-03-04T17:52:41.409 INFO:tasks.workunit.client.0.smithi137.stdout:release sdir_0 type: DIR refs: 2 2024-03-04T17:52:41.409 INFO:tasks.workunit.client.0.smithi137.stdout:release sfile_0 type: FILE refs: 2 2024-03-04T17:52:41.409 INFO:tasks.workunit.client.0.smithi137.stdout:release sdir_1 type: DIR refs: 2 2024-03-04T17:52:41.409 INFO:tasks.workunit.client.0.smithi137.stdout:release sfile_1 type: FILE refs: 2 2024-03-04T17:52:41.409 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-03-04T17:52:41.409 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.HIER1 2024-03-04T17:52:41.422 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/f2 (f2): FILE 2024-03-04T17:52:41.423 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/f1 (f1): FILE 2024-03-04T17:52:41.426 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2024-03-04T17:52:41.429 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2024-03-04T17:52:41.433 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-03-04T17:52:41.433 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-03-04T17:52:41.434 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2024-03-04T17:52:41.436 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2024-03-04T17:52:41.436 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-03-04T17:52:41.437 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-03-04T17:52:41.438 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2024-03-04T17:52:41.441 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2024-03-04T17:52:41.441 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz (baz): DIR 2024-03-04T17:52:41.441 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar (bar): DIR 2024-03-04T17:52:41.441 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo (foo): DIR 2024-03-04T17:52:41.442 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot (nfsroot): DIR 2024-03-04T17:52:41.442 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.HIER1 (33 ms) 2024-03-04T17:52:41.442 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-03-04T17:52:41.442 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-03-04T17:52:41.442 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-03-04T17:52:41.442 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-03-04T17:52:41.442 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-03-04T17:52:41.442 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-03-04T17:52:41.442 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-03-04T17:52:41.442 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-03-04T17:52:41.442 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:52:41.442 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:52:41.442 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:52:41.442 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:52:41.443 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:52:41.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:41.442+0000 7f16aefe3a80 -1 shutting down 2024-03-04T17:52:41.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:41.446+0000 7f158ab56640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 55811, si_uid : 1000, si_addr0x3e80000da03, si_status0 2024-03-04T17:52:41.526 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SHUTDOWN (85 ms) 2024-03-04T17:52:41.526 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 24 tests from LibRGW (1373 ms total) 2024-03-04T17:52:41.526 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:41.526 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:41.526 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 24 tests from 1 test suite ran. (1373 ms total) 2024-03-04T17:52:41.526 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 24 tests. 2024-03-04T17:52:41.541 INFO:tasks.workunit.client.0.smithi137.stdout:phase 1.2 2024-03-04T17:52:41.562 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 24 tests from 1 test suite. 2024-03-04T17:52:41.562 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:41.562 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 24 tests from LibRGW 2024-03-04T17:52:41.562 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:52:41.572 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:41.572 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:42.122 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.INIT (560 ms) 2024-03-04T17:52:42.122 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-03-04T17:52:42.123 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MOUNT_NOROOT (1 ms) 2024-03-04T17:52:42.123 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:52:42.123 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SETUP_HIER1 (0 ms) 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SETUP_DIRS1 (0 ms) 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SETATTR 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2024-03-04T17:52:42.124 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-03-04T17:52:42.125 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-03-04T17:52:42.125 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-03-04T17:52:42.125 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.READ_DIRS1 (0 ms) 2024-03-04T17:52:42.125 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-03-04T17:52:42.125 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2024-03-04T17:52:42.125 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-03-04T17:52:42.125 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-03-04T17:52:42.125 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-03-04T17:52:42.125 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-03-04T17:52:42.125 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-03-04T17:52:42.125 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-03-04T17:52:42.125 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.HIER1 2024-03-04T17:52:42.125 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.HIER1 (0 ms) 2024-03-04T17:52:42.125 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-03-04T17:52:42.126 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-03-04T17:52:42.126 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-03-04T17:52:42.126 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-03-04T17:52:42.126 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-03-04T17:52:42.126 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-03-04T17:52:42.126 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-03-04T17:52:42.126 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-03-04T17:52:42.126 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:52:42.126 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:52:42.126 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:52:42.126 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:52:42.126 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:52:42.127 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:42.126+0000 7fa2d1006a80 -1 shutting down 2024-03-04T17:52:42.127 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:42.126+0000 7fa1acb79640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 56461, si_uid : 1000, si_addr0x3e80000dc8d, si_status0 2024-03-04T17:52:42.284 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SHUTDOWN (161 ms) 2024-03-04T17:52:42.284 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 24 tests from LibRGW (722 ms total) 2024-03-04T17:52:42.284 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:42.284 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:42.284 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 24 tests from 1 test suite ran. (722 ms total) 2024-03-04T17:52:42.284 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 24 tests. 2024-03-04T17:52:42.298 INFO:tasks.workunit.client.0.smithi137.stdout:phase 1.3 2024-03-04T17:52:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 24 tests from 1 test suite. 2024-03-04T17:52:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 24 tests from LibRGW 2024-03-04T17:52:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:52:42.334 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:42.334 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:42.867 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.INIT (545 ms) 2024-03-04T17:52:42.867 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-03-04T17:52:42.867 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MOUNT_NOROOT (1 ms) 2024-03-04T17:52:42.868 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:52:42.868 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-03-04T17:52:42.868 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-03-04T17:52:42.877 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SETUP_HIER1 (9 ms) 2024-03-04T17:52:42.877 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-03-04T17:52:42.965 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SETUP_DIRS1 (88 ms) 2024-03-04T17:52:42.965 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SETATTR 2024-03-04T17:52:42.965 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-03-04T17:52:42.965 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-03-04T17:52:42.965 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2024-03-04T17:52:42.965 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-03-04T17:52:42.966 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2024-03-04T17:52:42.966 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-03-04T17:52:42.966 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2024-03-04T17:52:42.966 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-03-04T17:52:42.966 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2024-03-04T17:52:42.966 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-03-04T17:52:42.966 INFO:tasks.workunit.client.0.smithi137.stdout:scanning objects in /bdirs1/dir_0 2024-03-04T17:52:42.966 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-03-04T17:52:42.966 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-03-04T17:52:42.966 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-03-04T17:52:42.966 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-03-04T17:52:42.966 INFO:tasks.workunit.client.0.smithi137.stdout:scanning objects in /bdirs1/dir_1 2024-03-04T17:52:42.966 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-03-04T17:52:42.966 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-03-04T17:52:42.967 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-03-04T17:52:42.967 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-03-04T17:52:42.967 INFO:tasks.workunit.client.0.smithi137.stdout:scanning objects in /bdirs1/dir_2 2024-03-04T17:52:42.967 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-03-04T17:52:42.967 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-03-04T17:52:42.967 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-03-04T17:52:42.967 INFO:tasks.workunit.client.0.smithi137.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-03-04T17:52:42.967 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-03-04T17:52:42.967 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-03-04T17:52:42.967 INFO:tasks.workunit.client.0.smithi137.stdout:read back objects in /bdirs1/dir_0 2024-03-04T17:52:42.967 INFO:tasks.workunit.client.0.smithi137.stdout:reading 0,256 dir_0/sfile_0 2024-03-04T17:52:42.968 INFO:tasks.workunit.client.0.smithi137.stdout: read back from sfile_0 : "data for sfile_0" 2024-03-04T17:52:42.968 INFO:tasks.workunit.client.0.smithi137.stdout:reading 0,256 dir_0/sfile_1 2024-03-04T17:52:42.968 INFO:tasks.workunit.client.0.smithi137.stdout: read back from sfile_1 : "data for sfile_1" 2024-03-04T17:52:42.968 INFO:tasks.workunit.client.0.smithi137.stdout:read back objects in /bdirs1/dir_1 2024-03-04T17:52:42.968 INFO:tasks.workunit.client.0.smithi137.stdout:reading 0,256 dir_1/sfile_0 2024-03-04T17:52:42.968 INFO:tasks.workunit.client.0.smithi137.stdout: read back from sfile_0 : "data for sfile_0" 2024-03-04T17:52:42.969 INFO:tasks.workunit.client.0.smithi137.stdout:reading 0,256 dir_1/sfile_1 2024-03-04T17:52:42.971 INFO:tasks.workunit.client.0.smithi137.stdout: read back from sfile_1 : "data for sfile_1" 2024-03-04T17:52:42.971 INFO:tasks.workunit.client.0.smithi137.stdout:read back objects in /bdirs1/dir_2 2024-03-04T17:52:42.971 INFO:tasks.workunit.client.0.smithi137.stdout:reading 0,256 dir_2/sfile_0 2024-03-04T17:52:42.973 INFO:tasks.workunit.client.0.smithi137.stdout: read back from sfile_0 : "data for sfile_0" 2024-03-04T17:52:42.973 INFO:tasks.workunit.client.0.smithi137.stdout:reading 0,256 dir_2/sfile_1 2024-03-04T17:52:42.975 INFO:tasks.workunit.client.0.smithi137.stdout: read back from sfile_1 : "data for sfile_1" 2024-03-04T17:52:42.975 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.READ_DIRS1 (10 ms) 2024-03-04T17:52:42.975 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-03-04T17:52:42.975 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2024-03-04T17:52:42.975 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-03-04T17:52:42.976 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-03-04T17:52:42.976 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-03-04T17:52:42.976 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-03-04T17:52:42.976 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-03-04T17:52:42.976 INFO:tasks.workunit.client.0.smithi137.stdout:release sdir_0 type: DIR refs: 2 2024-03-04T17:52:42.976 INFO:tasks.workunit.client.0.smithi137.stdout:release sfile_0 type: FILE refs: 2 2024-03-04T17:52:42.976 INFO:tasks.workunit.client.0.smithi137.stdout:release sdir_1 type: DIR refs: 2 2024-03-04T17:52:42.976 INFO:tasks.workunit.client.0.smithi137.stdout:release sfile_1 type: FILE refs: 2 2024-03-04T17:52:42.976 INFO:tasks.workunit.client.0.smithi137.stdout:release sdir_0 type: DIR refs: 2 2024-03-04T17:52:42.976 INFO:tasks.workunit.client.0.smithi137.stdout:release sfile_0 type: FILE refs: 2 2024-03-04T17:52:42.976 INFO:tasks.workunit.client.0.smithi137.stdout:release sdir_1 type: DIR refs: 2 2024-03-04T17:52:42.976 INFO:tasks.workunit.client.0.smithi137.stdout:release sfile_1 type: FILE refs: 2 2024-03-04T17:52:42.977 INFO:tasks.workunit.client.0.smithi137.stdout:release sdir_0 type: DIR refs: 2 2024-03-04T17:52:42.977 INFO:tasks.workunit.client.0.smithi137.stdout:release sfile_0 type: FILE refs: 2 2024-03-04T17:52:42.977 INFO:tasks.workunit.client.0.smithi137.stdout:release sdir_1 type: DIR refs: 2 2024-03-04T17:52:42.977 INFO:tasks.workunit.client.0.smithi137.stdout:release sfile_1 type: FILE refs: 2 2024-03-04T17:52:42.977 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-03-04T17:52:42.977 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.HIER1 2024-03-04T17:52:42.987 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/f2 (f2): FILE 2024-03-04T17:52:42.988 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/f1 (f1): FILE 2024-03-04T17:52:42.990 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2024-03-04T17:52:42.992 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2024-03-04T17:52:42.995 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-03-04T17:52:42.996 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-03-04T17:52:42.997 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2024-03-04T17:52:42.998 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2024-03-04T17:52:42.999 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-03-04T17:52:43.000 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-03-04T17:52:43.001 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2024-03-04T17:52:43.003 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2024-03-04T17:52:43.004 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar/baz (baz): DIR 2024-03-04T17:52:43.004 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo/bar (bar): DIR 2024-03-04T17:52:43.004 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot/foo (foo): DIR 2024-03-04T17:52:43.004 INFO:tasks.workunit.client.0.smithi137.stdout:/nfsroot (nfsroot): DIR 2024-03-04T17:52:43.004 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.HIER1 (28 ms) 2024-03-04T17:52:43.004 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-03-04T17:52:43.004 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-03-04T17:52:43.004 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-03-04T17:52:43.004 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-03-04T17:52:43.004 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-03-04T17:52:43.004 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-03-04T17:52:43.004 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-03-04T17:52:43.004 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-03-04T17:52:43.004 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:52:43.005 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:52:43.005 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:52:43.005 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:52:43.005 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:52:43.005 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:43.006+0000 7fa8a54f8a80 -1 shutting down 2024-03-04T17:52:43.005 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:43.006+0000 7fa78106b640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 57104, si_uid : 1000, si_addr0x3e80000df10, si_status0 2024-03-04T17:52:43.113 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SHUTDOWN (110 ms) 2024-03-04T17:52:43.113 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 24 tests from LibRGW (792 ms total) 2024-03-04T17:52:43.113 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:43.113 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:43.114 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 24 tests from 1 test suite ran. (792 ms total) 2024-03-04T17:52:43.114 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 24 tests. 2024-03-04T17:52:43.124 INFO:tasks.workunit.client.0.smithi137.stdout:phase 2.1 2024-03-04T17:52:43.145 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 9 tests from 1 test suite. 2024-03-04T17:52:43.146 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:43.146 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 9 tests from LibRGW 2024-03-04T17:52:43.146 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:52:43.156 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:43.156 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:43.702 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.INIT (556 ms) 2024-03-04T17:52:43.702 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:52:43.702 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-03-04T17:52:43.703 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-03-04T17:52:43.718 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CREATE_BUCKET (15 ms) 2024-03-04T17:52:43.718 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-03-04T17:52:43.718 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-03-04T17:52:43.719 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2024-03-04T17:52:43.734 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave0 2024-03-04T17:52:43.748 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave1 2024-03-04T17:52:43.765 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave2 2024-03-04T17:52:43.784 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave3 2024-03-04T17:52:43.804 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave4 2024-03-04T17:52:43.826 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave5 2024-03-04T17:52:43.847 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave6 2024-03-04T17:52:43.871 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave7 2024-03-04T17:52:43.896 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave8 2024-03-04T17:52:43.922 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave9 2024-03-04T17:52:43.922 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (204 ms) 2024-03-04T17:52:43.922 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2024-03-04T17:52:44.392 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (470 ms) 2024-03-04T17:52:44.392 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:52:44.392 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:52:44.392 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:52:44.392 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:52:44.393 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:52:44.393 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:44.394+0000 7f5b4e82da80 -1 shutting down 2024-03-04T17:52:44.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:44.398+0000 7f5a2a3a0640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 57754, si_uid : 1000, si_addr0x3e80000e19a, si_status0 2024-03-04T17:52:44.555 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SHUTDOWN (163 ms) 2024-03-04T17:52:44.555 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 9 tests from LibRGW (1409 ms total) 2024-03-04T17:52:44.555 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:44.556 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:44.556 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 9 tests from 1 test suite ran. (1409 ms total) 2024-03-04T17:52:44.556 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 9 tests. 2024-03-04T17:52:44.569 INFO:tasks.workunit.client.0.smithi137.stdout:phase 2.2 2024-03-04T17:52:44.591 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 9 tests from 1 test suite. 2024-03-04T17:52:44.591 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:44.591 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 9 tests from LibRGW 2024-03-04T17:52:44.591 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:52:44.602 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:44.602 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:44.870 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.INIT (279 ms) 2024-03-04T17:52:44.870 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:52:44.870 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-03-04T17:52:44.870 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-03-04T17:52:44.870 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-03-04T17:52:44.870 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-03-04T17:52:44.900 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.DELETE_BUCKET (30 ms) 2024-03-04T17:52:44.900 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2024-03-04T17:52:44.917 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave0 2024-03-04T17:52:44.932 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave1 2024-03-04T17:52:44.949 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave2 2024-03-04T17:52:44.968 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave3 2024-03-04T17:52:44.987 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave4 2024-03-04T17:52:45.008 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave5 2024-03-04T17:52:45.026 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave6 2024-03-04T17:52:45.047 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave7 2024-03-04T17:52:45.067 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave8 2024-03-04T17:52:45.090 INFO:tasks.workunit.client.0.smithi137.stdout:created: sorrydave9 2024-03-04T17:52:45.090 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (189 ms) 2024-03-04T17:52:45.090 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2024-03-04T17:52:45.559 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (469 ms) 2024-03-04T17:52:45.559 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:52:45.559 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:52:45.559 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:52:45.559 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:52:45.559 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:52:45.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:45.562+0000 7fa980205a80 -1 shutting down 2024-03-04T17:52:45.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:45.562+0000 7fa85bd78640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 58397, si_uid : 1000, si_addr0x3e80000e41d, si_status0 2024-03-04T17:52:45.723 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SHUTDOWN (164 ms) 2024-03-04T17:52:45.723 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 9 tests from LibRGW (1132 ms total) 2024-03-04T17:52:45.723 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:45.724 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:45.724 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 9 tests from 1 test suite ran. (1132 ms total) 2024-03-04T17:52:45.724 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 9 tests. 2024-03-04T17:52:45.737 INFO:tasks.workunit.client.0.smithi137.stdout:phase 3.1 2024-03-04T17:52:46.078 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 13 tests from 1 test suite. 2024-03-04T17:52:46.078 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:46.079 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 13 tests from LibRGW 2024-03-04T17:52:46.079 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:52:46.088 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:46.088 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:46.649 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.INIT (571 ms) 2024-03-04T17:52:46.649 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:52:46.650 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-03-04T17:52:46.650 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-03-04T17:52:46.667 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CREATE_BUCKET (17 ms) 2024-03-04T17:52:46.667 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-03-04T17:52:46.674 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (7 ms) 2024-03-04T17:52:46.674 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-03-04T17:52:46.676 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (3 ms) 2024-03-04T17:52:46.676 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.LARGE1 2024-03-04T17:52:46.788 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.LARGE1 (111 ms) 2024-03-04T17:52:46.788 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.LARGE2 2024-03-04T17:52:46.871 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.LARGE2 (83 ms) 2024-03-04T17:52:46.871 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-03-04T17:52:46.871 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-03-04T17:52:46.871 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-03-04T17:52:46.871 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2024-03-04T17:52:46.871 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-03-04T17:52:46.872 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-03-04T17:52:46.872 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:52:46.872 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:52:46.872 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:52:46.872 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:52:46.872 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:52:46.872 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:46.874+0000 7f7ef1d9ea80 -1 shutting down 2024-03-04T17:52:46.874 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:46.874+0000 7f7dcd911640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 59040, si_uid : 1000, si_addr0x3e80000e6a0, si_status0 2024-03-04T17:52:47.010 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SHUTDOWN (139 ms) 2024-03-04T17:52:47.010 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 13 tests from LibRGW (932 ms total) 2024-03-04T17:52:47.010 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:47.010 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:47.010 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 13 tests from 1 test suite ran. (932 ms total) 2024-03-04T17:52:47.010 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 13 tests. 2024-03-04T17:52:47.028 INFO:tasks.workunit.client.0.smithi137.stdout:phase 3.2 2024-03-04T17:52:47.376 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 13 tests from 1 test suite. 2024-03-04T17:52:47.376 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:47.376 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 13 tests from LibRGW 2024-03-04T17:52:47.376 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:52:47.385 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:47.385 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:48.046 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.INIT (670 ms) 2024-03-04T17:52:48.046 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:52:48.047 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-03-04T17:52:48.047 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-03-04T17:52:48.047 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-03-04T17:52:48.047 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-03-04T17:52:48.059 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (12 ms) 2024-03-04T17:52:48.059 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-03-04T17:52:48.066 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (7 ms) 2024-03-04T17:52:48.066 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.LARGE1 2024-03-04T17:52:48.184 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.LARGE1 (119 ms) 2024-03-04T17:52:48.185 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.LARGE2 2024-03-04T17:52:48.185 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.LARGE2 (0 ms) 2024-03-04T17:52:48.185 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-03-04T17:52:48.185 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-03-04T17:52:48.185 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-03-04T17:52:48.198 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.DELETE_OBJECT (13 ms) 2024-03-04T17:52:48.198 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-03-04T17:52:48.246 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.DELETE_BUCKET (48 ms) 2024-03-04T17:52:48.246 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:52:48.246 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:52:48.246 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:52:48.246 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:52:48.246 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:52:48.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:48.246+0000 7f7299e65a80 -1 shutting down 2024-03-04T17:52:48.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:48.250+0000 7f71759d8640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 59683, si_uid : 1000, si_addr0x3e80000e923, si_status0 2024-03-04T17:52:48.427 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SHUTDOWN (181 ms) 2024-03-04T17:52:48.427 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 13 tests from LibRGW (1051 ms total) 2024-03-04T17:52:48.427 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:48.427 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:48.427 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 13 tests from 1 test suite ran. (1051 ms total) 2024-03-04T17:52:48.427 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 13 tests. 2024-03-04T17:52:48.447 INFO:tasks.workunit.client.0.smithi137.stdout:phase 4.1 2024-03-04T17:52:48.470 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 11 tests from 1 test suite. 2024-03-04T17:52:48.470 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:48.470 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 11 tests from LibRGW 2024-03-04T17:52:48.470 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.TVAR 2024-03-04T17:52:48.470 INFO:tasks.workunit.client.0.smithi137.stdout:read i1: 64001 2024-03-04T17:52:48.470 INFO:tasks.workunit.client.0.smithi137.stdout:read s1: blunderbuss 2024-03-04T17:52:48.470 INFO:tasks.workunit.client.0.smithi137.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2024-03-04T17:52:48.470 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:52:48.483 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:48.483 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:49.015 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.INIT (545 ms) 2024-03-04T17:52:49.016 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:52:49.016 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-03-04T17:52:49.016 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-03-04T17:52:49.037 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (22 ms) 2024-03-04T17:52:49.038 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-03-04T17:52:49.848 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_0 2024-03-04T17:52:49.848 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_1 2024-03-04T17:52:49.848 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_2 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_3 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_4 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_5 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_6 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_7 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_8 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_9 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_10 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_11 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_12 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_13 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_14 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_15 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_16 2024-03-04T17:52:49.849 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_17 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_18 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_19 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_20 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_21 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_22 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_23 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_24 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_25 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_26 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_27 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_28 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_29 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_30 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_31 2024-03-04T17:52:49.850 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_32 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_33 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_34 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_35 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_36 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_37 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_38 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_39 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_40 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_41 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_42 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_43 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_44 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_45 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_46 2024-03-04T17:52:49.851 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_47 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_48 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_49 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_50 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_51 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_52 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_53 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_54 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_55 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_56 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_57 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_58 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_59 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_60 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_61 2024-03-04T17:52:49.852 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_62 2024-03-04T17:52:49.853 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_63 2024-03-04T17:52:49.853 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_64 2024-03-04T17:52:49.853 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_65 2024-03-04T17:52:49.853 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_66 2024-03-04T17:52:49.853 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_67 2024-03-04T17:52:49.853 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_68 2024-03-04T17:52:49.853 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_69 2024-03-04T17:52:49.853 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_70 2024-03-04T17:52:49.853 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_71 2024-03-04T17:52:49.853 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_72 2024-03-04T17:52:49.853 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_73 2024-03-04T17:52:49.853 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_74 2024-03-04T17:52:49.853 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_75 2024-03-04T17:52:49.853 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_76 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_77 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_78 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_79 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_80 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_81 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_82 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_83 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_84 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_85 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_86 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_87 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_88 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_89 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_90 2024-03-04T17:52:49.854 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_91 2024-03-04T17:52:49.855 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_92 2024-03-04T17:52:49.855 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_93 2024-03-04T17:52:49.855 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_94 2024-03-04T17:52:49.855 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_95 2024-03-04T17:52:49.855 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_96 2024-03-04T17:52:49.855 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_97 2024-03-04T17:52:49.855 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_98 2024-03-04T17:52:49.855 INFO:tasks.workunit.client.0.smithi137.stdout:created: dmarker:f_99 2024-03-04T17:52:49.855 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (810 ms) 2024-03-04T17:52:49.855 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-03-04T17:52:49.856 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=. flags=4 2024-03-04T17:52:49.856 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=.. flags=4 2024-03-04T17:52:49.856 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-03-04T17:52:49.856 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-03-04T17:52:49.856 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-03-04T17:52:49.856 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-03-04T17:52:49.856 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-03-04T17:52:49.856 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-03-04T17:52:49.856 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-03-04T17:52:49.856 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-03-04T17:52:49.856 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-03-04T17:52:49.857 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-03-04T17:52:49.857 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-03-04T17:52:49.857 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-03-04T17:52:49.857 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-03-04T17:52:49.857 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-03-04T17:52:49.857 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-03-04T17:52:49.857 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-03-04T17:52:49.857 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-03-04T17:52:49.857 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-03-04T17:52:49.857 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-03-04T17:52:49.857 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-03-04T17:52:49.857 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-03-04T17:52:49.857 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-03-04T17:52:49.857 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-03-04T17:52:49.858 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-03-04T17:52:49.858 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-03-04T17:52:49.858 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-03-04T17:52:49.858 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-03-04T17:52:49.858 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-03-04T17:52:49.858 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-03-04T17:52:49.858 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-03-04T17:52:49.858 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-03-04T17:52:49.858 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-03-04T17:52:49.858 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-03-04T17:52:49.858 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-03-04T17:52:49.858 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-03-04T17:52:49.858 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-03-04T17:52:49.858 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-03-04T17:52:49.859 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-03-04T17:52:49.860 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-03-04T17:52:49.860 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-03-04T17:52:49.860 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-03-04T17:52:49.860 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-03-04T17:52:49.860 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-03-04T17:52:49.860 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-03-04T17:52:49.860 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-03-04T17:52:49.860 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-03-04T17:52:49.860 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-03-04T17:52:49.860 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-03-04T17:52:49.860 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-03-04T17:52:49.860 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-03-04T17:52:49.860 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-03-04T17:52:49.860 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-03-04T17:52:49.861 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-03-04T17:52:49.862 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-03-04T17:52:49.863 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-03-04T17:52:49.863 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-03-04T17:52:49.863 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-03-04T17:52:49.863 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_READDIR (8 ms) 2024-03-04T17:52:49.863 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2024-03-04T17:52:49.864 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-03-04T17:52:49.864 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-03-04T17:52:49.864 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-03-04T17:52:49.864 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-03-04T17:52:49.864 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-03-04T17:52:49.864 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-03-04T17:52:49.864 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-03-04T17:52:49.864 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-03-04T17:52:49.864 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-03-04T17:52:49.864 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-03-04T17:52:49.864 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-03-04T17:52:49.865 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-03-04T17:52:49.866 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-03-04T17:52:49.867 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-03-04T17:52:49.868 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-03-04T17:52:49.869 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-03-04T17:52:49.870 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-03-04T17:52:49.870 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-03-04T17:52:49.870 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-03-04T17:52:49.870 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-03-04T17:52:49.870 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-03-04T17:52:49.870 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-03-04T17:52:49.870 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-03-04T17:52:49.870 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-03-04T17:52:49.870 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-03-04T17:52:49.870 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-03-04T17:52:49.870 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-03-04T17:52:49.870 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-03-04T17:52:49.870 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-03-04T17:52:49.870 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER2_READDIR (5 ms) 2024-03-04T17:52:49.871 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-03-04T17:52:49.871 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-03-04T17:52:49.871 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:52:49.871 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:52:49.871 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:52:49.871 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:52:49.871 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:52:49.871 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:49.862+0000 7f58930bfa80 -1 shutting down 2024-03-04T17:52:49.871 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:49.866+0000 7f576ec32640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 60326, si_uid : 1000, si_addr0x3e80000eba6, si_status0 2024-03-04T17:52:49.982 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SHUTDOWN (121 ms) 2024-03-04T17:52:49.982 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 11 tests from LibRGW (1512 ms total) 2024-03-04T17:52:49.982 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:49.982 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:49.982 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 11 tests from 1 test suite ran. (1512 ms total) 2024-03-04T17:52:49.983 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 11 tests. 2024-03-04T17:52:49.998 INFO:tasks.workunit.client.0.smithi137.stdout:phase 4.2 2024-03-04T17:52:50.023 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 11 tests from 1 test suite. 2024-03-04T17:52:50.023 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:50.023 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 11 tests from LibRGW 2024-03-04T17:52:50.023 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.TVAR 2024-03-04T17:52:50.023 INFO:tasks.workunit.client.0.smithi137.stdout:read i1: 64001 2024-03-04T17:52:50.023 INFO:tasks.workunit.client.0.smithi137.stdout:read s1: blunderbuss 2024-03-04T17:52:50.023 INFO:tasks.workunit.client.0.smithi137.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2024-03-04T17:52:50.023 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:52:50.032 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:50.032 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:50.580 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.INIT (557 ms) 2024-03-04T17:52:50.580 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:52:50.581 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-03-04T17:52:50.581 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-03-04T17:52:50.595 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (14 ms) 2024-03-04T17:52:50.595 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-03-04T17:52:50.595 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-03-04T17:52:50.595 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-03-04T17:52:50.595 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-03-04T17:52:50.595 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2024-03-04T17:52:50.599 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-03-04T17:52:50.599 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-03-04T17:52:50.600 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-03-04T17:52:50.600 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-03-04T17:52:50.600 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-03-04T17:52:50.600 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-03-04T17:52:50.600 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-03-04T17:52:50.600 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-03-04T17:52:50.600 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-03-04T17:52:50.600 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-03-04T17:52:50.600 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-03-04T17:52:50.600 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-03-04T17:52:50.600 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-03-04T17:52:50.600 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-03-04T17:52:50.600 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-03-04T17:52:50.600 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-03-04T17:52:50.601 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-03-04T17:52:50.602 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-03-04T17:52:50.603 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-03-04T17:52:50.604 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-03-04T17:52:50.605 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-03-04T17:52:50.606 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-03-04T17:52:50.606 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-03-04T17:52:50.606 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-03-04T17:52:50.606 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-03-04T17:52:50.606 INFO:tasks.workunit.client.0.smithi137.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-03-04T17:52:50.606 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER2_READDIR (4 ms) 2024-03-04T17:52:50.606 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-03-04T17:52:50.606 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-03-04T17:52:50.606 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:52:50.606 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:52:50.606 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:52:50.606 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:52:50.606 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:52:50.607 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:50.602+0000 7f1c72fffa80 -1 shutting down 2024-03-04T17:52:50.607 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:50.602+0000 7f1b4eb72640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 60969, si_uid : 1000, si_addr0x3e80000ee29, si_status0 2024-03-04T17:52:50.739 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SHUTDOWN (140 ms) 2024-03-04T17:52:50.739 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 11 tests from LibRGW (716 ms total) 2024-03-04T17:52:50.739 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:50.739 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:50.739 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 11 tests from 1 test suite ran. (716 ms total) 2024-03-04T17:52:50.740 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 11 tests. 2024-03-04T17:52:50.753 INFO:tasks.workunit.client.0.smithi137.stdout:phase 5.1 2024-03-04T17:52:50.786 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 19 tests from 1 test suite. 2024-03-04T17:52:50.786 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:50.786 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 19 tests from LibRGW 2024-03-04T17:52:50.786 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:52:50.797 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:50.797 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:51.341 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.INIT (556 ms) 2024-03-04T17:52:51.342 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:52:51.342 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-03-04T17:52:51.342 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-03-04T17:52:51.358 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CREATE_BUCKET (16 ms) 2024-03-04T17:52:51.358 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-03-04T17:52:51.363 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (4 ms) 2024-03-04T17:52:51.363 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2024-03-04T17:52:51.363 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2024-03-04T17:52:51.363 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-03-04T17:52:51.365 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (2 ms) 2024-03-04T17:52:51.365 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.OBJ_OPEN 2024-03-04T17:52:51.365 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2024-03-04T17:52:51.365 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.PUT_OBJECT 2024-03-04T17:52:51.369 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.PUT_OBJECT (5 ms) 2024-03-04T17:52:51.369 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.GET_OBJECT 2024-03-04T17:52:51.372 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.GET_OBJECT (2 ms) 2024-03-04T17:52:51.372 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-03-04T17:52:51.372 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-03-04T17:52:51.372 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2024-03-04T17:52:51.372 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.WRITEV 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.READV 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.READV (0 ms) 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:52:51.373 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:52:51.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:51.374+0000 7fa4bf984a80 -1 shutting down 2024-03-04T17:52:51.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:51.374+0000 7fa39b4f7640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 61612, si_uid : 1000, si_addr0x3e80000f0ac, si_status0 2024-03-04T17:52:51.500 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SHUTDOWN (129 ms) 2024-03-04T17:52:51.501 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 19 tests from LibRGW (715 ms total) 2024-03-04T17:52:51.501 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:51.501 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:51.501 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 19 tests from 1 test suite ran. (715 ms total) 2024-03-04T17:52:51.501 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 19 tests. 2024-03-04T17:52:51.513 INFO:tasks.workunit.client.0.smithi137.stdout:phase 5.2 2024-03-04T17:52:51.546 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] Running 19 tests from 1 test suite. 2024-03-04T17:52:51.546 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment set-up. 2024-03-04T17:52:51.546 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 19 tests from LibRGW 2024-03-04T17:52:51.546 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.INIT 2024-03-04T17:52:51.554 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:51.554 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:52.080 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.INIT (534 ms) 2024-03-04T17:52:52.080 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.MOUNT 2024-03-04T17:52:52.080 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-03-04T17:52:52.080 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-03-04T17:52:52.081 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-03-04T17:52:52.081 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-03-04T17:52:52.087 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (6 ms) 2024-03-04T17:52:52.087 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2024-03-04T17:52:52.087 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2024-03-04T17:52:52.087 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-03-04T17:52:52.087 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (0 ms) 2024-03-04T17:52:52.087 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.OBJ_OPEN 2024-03-04T17:52:52.087 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2024-03-04T17:52:52.088 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.PUT_OBJECT 2024-03-04T17:52:52.088 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.PUT_OBJECT (0 ms) 2024-03-04T17:52:52.088 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.GET_OBJECT 2024-03-04T17:52:52.088 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.GET_OBJECT (0 ms) 2024-03-04T17:52:52.088 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-03-04T17:52:52.088 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-03-04T17:52:52.088 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2024-03-04T17:52:52.088 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2024-03-04T17:52:52.088 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.WRITEV 2024-03-04T17:52:52.088 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2024-03-04T17:52:52.088 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.READV 2024-03-04T17:52:52.088 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.READV (0 ms) 2024-03-04T17:52:52.088 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2024-03-04T17:52:52.088 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2024-03-04T17:52:52.089 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-03-04T17:52:52.091 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.DELETE_OBJECT (4 ms) 2024-03-04T17:52:52.091 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-03-04T17:52:52.119 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.DELETE_BUCKET (28 ms) 2024-03-04T17:52:52.119 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.CLEANUP 2024-03-04T17:52:52.119 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-03-04T17:52:52.119 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.UMOUNT 2024-03-04T17:52:52.119 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-03-04T17:52:52.119 INFO:tasks.workunit.client.0.smithi137.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-03-04T17:52:52.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.122+0000 7f7eece2ca80 -1 shutting down 2024-03-04T17:52:52.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.122+0000 7f7dc899f640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 62257, si_uid : 1000, si_addr0x3e80000f331, si_status0 2024-03-04T17:52:52.242 INFO:tasks.workunit.client.0.smithi137.stdout:[ OK ] LibRGW.SHUTDOWN (123 ms) 2024-03-04T17:52:52.242 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] 19 tests from LibRGW (696 ms total) 2024-03-04T17:52:52.242 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:52.242 INFO:tasks.workunit.client.0.smithi137.stdout:[----------] Global test environment tear-down 2024-03-04T17:52:52.242 INFO:tasks.workunit.client.0.smithi137.stdout:[==========] 19 tests from 1 test suite ran. (696 ms total) 2024-03-04T17:52:52.242 INFO:tasks.workunit.client.0.smithi137.stdout:[ PASSED ] 19 tests. 2024-03-04T17:52:52.257 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:52:52.257 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:52:52.273 INFO:tasks.workunit:Running workunits matching rgw/test_awssdkv4_sig.sh on client.0... 2024-03-04T17:52:52.274 INFO:tasks.workunit:Running workunit rgw/test_awssdkv4_sig.sh... 2024-03-04T17:52:52.274 DEBUG:teuthology.orchestra.run.smithi137:workunit test rgw/test_awssdkv4_sig.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_awssdkv4_sig.sh 2024-03-04T17:52:52.323 INFO:tasks.workunit.client.0.smithi137.stderr:+ [ -z ] 2024-03-04T17:52:52.323 INFO:tasks.workunit.client.0.smithi137.stderr:+ export AWS_ACCESS_KEY_ID=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-03-04T17:52:52.323 INFO:tasks.workunit.client.0.smithi137.stderr:+ export AWS_SECRET_ACCESS_KEY=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-03-04T17:52:52.323 INFO:tasks.workunit.client.0.smithi137.stderr:+ radosgw-admin user create --uid ceph-test-maven --access-key lNCnR47C2g+ZidCWBAUuwfSAA7Q= --secret tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== --display-name maven test user --email sigv4@example.com 2024-03-04T17:52:52.363 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:52:52.363 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:52:52.405 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.406+0000 7f3424faf980 20 rados->read ofs=0 len=0 2024-03-04T17:52:52.407 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.410+0000 7f3424faf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:52.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.410+0000 7f3424faf980 20 realm 2024-03-04T17:52:52.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.410+0000 7f3424faf980 20 rados->read ofs=0 len=0 2024-03-04T17:52:52.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.410+0000 7f3424faf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:52.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.410+0000 7f3424faf980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:52:52.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.410+0000 7f3424faf980 20 rados->read ofs=0 len=0 2024-03-04T17:52:52.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.410+0000 7f3424faf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:52.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.410+0000 7f3424faf980 20 rados->read ofs=0 len=0 2024-03-04T17:52:52.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.410+0000 7f3424faf980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:52:52.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.410+0000 7f3424faf980 20 rados->read ofs=0 len=0 2024-03-04T17:52:52.410 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.410+0000 7f3424faf980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:52:52.410 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.410+0000 7f3424faf980 20 searching for the correct realm 2024-03-04T17:52:52.413 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.414+0000 7f3422ac7640 0 -- 172.21.15.137:0/2970460587 >> [v2:172.21.15.137:6824/1039931256,v1:172.21.15.137:6825/1039931256] conn(0x55ce873d55a0 msgr2=0x55ce873d79b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:52:52.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.422+0000 7f3422ac7640 0 -- 172.21.15.137:0/2970460587 >> [v2:172.21.15.173:6805/879938785,v1:172.21.15.173:6808/879938785] conn(0x7f3414018070 msgr2=0x7f3414058410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:52:52.639 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:52:52.639 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:52:52.639 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:52:52.639 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:52:52.639 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:52:52.639 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:52:52.640 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 rados->read ofs=0 len=0 2024-03-04T17:52:52.640 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:52.640 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 rados->read ofs=0 len=0 2024-03-04T17:52:52.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:52:52.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 rados->read ofs=0 len=0 2024-03-04T17:52:52.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:52:52.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 zone default found 2024-03-04T17:52:52.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 4 Realm: () 2024-03-04T17:52:52.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:52:52.641 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:52:52.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:52:52.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 zonegroup default 2024-03-04T17:52:52.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 rados->read ofs=0 len=0 2024-03-04T17:52:52.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:52.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.642+0000 7f3424faf980 20 rados->read ofs=0 len=0 2024-03-04T17:52:52.643 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.646+0000 7f3424faf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:52.643 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.646+0000 7f3424faf980 20 rados->read ofs=0 len=0 2024-03-04T17:52:52.643 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.646+0000 7f3424faf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:52.643 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.646+0000 7f3424faf980 20 started sync module instance, tier type = 2024-03-04T17:52:52.643 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.646+0000 7f3424faf980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:52:52.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.666+0000 7f3424faf980 20 add_watcher() i=0 2024-03-04T17:52:52.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.666+0000 7f3424faf980 20 add_watcher() i=1 2024-03-04T17:52:52.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.666+0000 7f3424faf980 20 add_watcher() i=2 2024-03-04T17:52:52.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.666+0000 7f3424faf980 20 add_watcher() i=3 2024-03-04T17:52:52.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.666+0000 7f3424faf980 20 add_watcher() i=4 2024-03-04T17:52:52.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.666+0000 7f3424faf980 20 add_watcher() i=5 2024-03-04T17:52:52.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.666+0000 7f3424faf980 20 add_watcher() i=6 2024-03-04T17:52:52.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.670+0000 7f3424faf980 20 add_watcher() i=7 2024-03-04T17:52:52.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.670+0000 7f3424faf980 2 all 8 watchers are set, enabling cache 2024-03-04T17:52:52.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.674+0000 7f32d57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:52:52.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.674+0000 7f3424faf980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:52:52.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.674+0000 7f3424faf980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:52:52.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 5 note: GC not initialized 2024-03-04T17:52:52.675 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f324ffff640 20 reqs_thread_entry: start 2024-03-04T17:52:52.675 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 20 init_complete bucket index max shards: 1 2024-03-04T17:52:52.675 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 20 Filter name: none 2024-03-04T17:52:52.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f324dffb640 20 reqs_thread_entry: start 2024-03-04T17:52:52.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 10 cache get: name=default.rgw.meta+users.uid+ceph-test-maven : miss 2024-03-04T17:52:52.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 20 rados->read ofs=0 len=0 2024-03-04T17:52:52.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:52.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x0 2024-03-04T17:52:52.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 10 adding default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-03-04T17:52:52.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 10 cache get: name=default.rgw.meta+users.email+sigv4@example.com : miss 2024-03-04T17:52:52.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 20 rados->read ofs=0 len=0 2024-03-04T17:52:52.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:52.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x0 2024-03-04T17:52:52.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 10 adding default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-03-04T17:52:52.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : miss 2024-03-04T17:52:52.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 20 rados->read ofs=0 len=0 2024-03-04T17:52:52.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:52:52.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x0 2024-03-04T17:52:52.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 10 adding default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-03-04T17:52:52.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2024-03-04T17:52:52.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.678+0000 7f3424faf980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2024-03-04T17:52:52.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.682+0000 7f3424faf980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2024-03-04T17:52:52.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.682+0000 7f3424faf980 10 moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-03-04T17:52:52.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.682+0000 7f3424faf980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-maven, ofs0, ns] 2024-03-04T17:52:52.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.686+0000 7f32fe7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1060856922121 cookie 94345520633120 notifier 5465 bl.length()=644 2024-03-04T17:52:52.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.686+0000 7f32fe7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2024-03-04T17:52:52.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.686+0000 7f32fe7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-03-04T17:52:52.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.690+0000 7f3424faf980 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2024-03-04T17:52:52.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.690+0000 7f3424faf980 10 moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-03-04T17:52:52.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.690+0000 7f3424faf980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:sigv4@example.com, ofs0, ns] 2024-03-04T17:52:52.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.690+0000 7f32fe7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1060856922122 cookie 94345520617792 notifier 5465 bl.length()=187 2024-03-04T17:52:52.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.690+0000 7f32fe7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2024-03-04T17:52:52.689 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.690+0000 7f32fe7fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-03-04T17:52:52.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.694+0000 7f3424faf980 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2024-03-04T17:52:52.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.694+0000 7f3424faf980 10 moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-03-04T17:52:52.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.694+0000 7f3424faf980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:lNCnR47C2g+ZidCWBAUuwfSAA7Q=, ofs0, ns] 2024-03-04T17:52:52.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.698+0000 7f32fe7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1060856922123 cookie 94345520617792 notifier 5465 bl.length()=197 2024-03-04T17:52:52.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.698+0000 7f32fe7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2024-03-04T17:52:52.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.698+0000 7f32fe7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-03-04T17:52:52.697 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:52:52.697 INFO:tasks.workunit.client.0.smithi137.stdout: "user_id": "ceph-test-maven", 2024-03-04T17:52:52.697 INFO:tasks.workunit.client.0.smithi137.stdout: "display_name": "maven test user", 2024-03-04T17:52:52.697 INFO:tasks.workunit.client.0.smithi137.stdout: "email": "sigv4@example.com", 2024-03-04T17:52:52.697 INFO:tasks.workunit.client.0.smithi137.stdout: "suspended": 0, 2024-03-04T17:52:52.697 INFO:tasks.workunit.client.0.smithi137.stdout: "max_buckets": 1000, 2024-03-04T17:52:52.697 INFO:tasks.workunit.client.0.smithi137.stdout: "subusers": [], 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: "keys": [ 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: { 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: "user": "ceph-test-maven", 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: "access_key": "lNCnR47C2g+ZidCWBAUuwfSAA7Q=", 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: "secret_key": "tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g==", 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: "active": true 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: ], 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: "swift_keys": [], 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: "caps": [], 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: "op_mask": "read, write, delete", 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: "default_placement": "", 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: "default_storage_class": "", 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: "placement_tags": [], 2024-03-04T17:52:52.698 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_quota": { 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: "enabled": false, 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: "check_on_raw": false, 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: "max_size": -1, 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: "max_size_kb": 0, 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: "max_objects": -1 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: "user_quota": { 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: "enabled": false, 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: "check_on_raw": false, 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: "max_size": -1, 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: "max_size_kb": 0, 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: "max_objects": -1 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: "temp_url_keys": [], 2024-03-04T17:52:52.699 INFO:tasks.workunit.client.0.smithi137.stdout: "type": "rgw", 2024-03-04T17:52:52.700 INFO:tasks.workunit.client.0.smithi137.stdout: "mfa_ids": [] 2024-03-04T17:52:52.700 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:52:52.700 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:52:52.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.698+0000 7f32d57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:52:52.703 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.706+0000 7f3424faf980 20 remove_watcher() i=0 2024-03-04T17:52:52.703 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.706+0000 7f3424faf980 2 removed watcher, disabling cache 2024-03-04T17:52:52.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.706+0000 7f3424faf980 20 remove_watcher() i=1 2024-03-04T17:52:52.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.710+0000 7f3424faf980 20 remove_watcher() i=2 2024-03-04T17:52:52.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.710+0000 7f3424faf980 20 remove_watcher() i=3 2024-03-04T17:52:52.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.714+0000 7f3424faf980 20 remove_watcher() i=4 2024-03-04T17:52:52.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.714+0000 7f3424faf980 20 remove_watcher() i=5 2024-03-04T17:52:52.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.718+0000 7f3424faf980 20 remove_watcher() i=6 2024-03-04T17:52:52.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T17:52:52.722+0000 7f3424faf980 20 remove_watcher() i=7 2024-03-04T17:52:52.737 INFO:tasks.workunit.client.0.smithi137.stderr:+ [ -z ] 2024-03-04T17:52:52.737 INFO:tasks.workunit.client.0.smithi137.stderr:+ [ -z /home/ubuntu/cephtest ] 2024-03-04T17:52:52.737 INFO:tasks.workunit.client.0.smithi137.stderr:+ cat /home/ubuntu/cephtest/url_file 2024-03-04T17:52:52.738 INFO:tasks.workunit.client.0.smithi137.stderr:+ export RGW_HTTP_ENDPOINT_URL=https://smithi137.front.sepia.ceph.com:443 2024-03-04T17:52:52.738 INFO:tasks.workunit.client.0.smithi137.stderr:+ [ -z /home/ubuntu/cephtest/clone.client.0 ] 2024-03-04T17:52:52.738 INFO:tasks.workunit.client.0.smithi137.stderr:+ cd /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum 2024-03-04T17:52:52.738 INFO:tasks.workunit.client.0.smithi137.stderr:+ ./mvnw clean package 2024-03-04T17:52:52.745 INFO:tasks.workunit.client.0.smithi137.stdout:Warning: JAVA_HOME environment variable is not set. 2024-03-04T17:52:56.688 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Scanning for projects... 2024-03-04T17:52:56.844 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom 2024-03-04T17:52:57.018 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.4/5.6 kB Progress (1): 2.8/5.6 kB Progress (1): 4.1/5.6 kB Progress (1): 5.5/5.6 kB Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom (5.6 kB at 32 kB/s) 2024-03-04T17:52:57.037 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom 2024-03-04T17:52:57.121 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/83 kB Progress (1): 33/83 kB Progress (1): 49/83 kB Progress (1): 66/83 kB Progress (1): 82/83 kB Progress (1): 83 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom (83 kB at 987 kB/s) 2024-03-04T17:52:57.150 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom 2024-03-04T17:52:57.196 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/77 kB Progress (1): 33/77 kB Progress (1): 49/77 kB Progress (1): 66/77 kB Progress (1): 77 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom (77 kB at 1.7 MB/s) 2024-03-04T17:52:57.253 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:52:57.253 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2024-03-04T17:52:57.253 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Building jcksum 1.0 2024-03-04T17:52:57.253 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] from pom.xml 2024-03-04T17:52:57.253 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2024-03-04T17:52:57.259 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom 2024-03-04T17:52:57.282 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom (5.3 kB at 231 kB/s) 2024-03-04T17:52:57.290 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom 2024-03-04T17:52:57.312 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom (9.9 kB at 450 kB/s) 2024-03-04T17:52:57.318 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom 2024-03-04T17:52:57.343 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/45 kB Progress (1): 33/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom (45 kB at 1.8 MB/s) 2024-03-04T17:52:57.355 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom 2024-03-04T17:52:57.377 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom (21 kB at 934 kB/s) 2024-03-04T17:52:57.398 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar 2024-03-04T17:52:57.420 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/36 kB Progress (1): 33/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar (36 kB at 1.6 MB/s) 2024-03-04T17:52:57.434 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom 2024-03-04T17:52:57.453 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom (8.2 kB at 429 kB/s) 2024-03-04T17:52:57.459 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom 2024-03-04T17:52:57.478 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom (8.1 kB at 405 kB/s) 2024-03-04T17:52:57.483 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom 2024-03-04T17:52:57.504 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/48 kB Progress (1): 33/48 kB Progress (1): 48 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom (48 kB at 2.2 MB/s) 2024-03-04T17:52:57.512 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom 2024-03-04T17:52:57.532 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom (21 kB at 1.0 MB/s) 2024-03-04T17:52:57.543 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar 2024-03-04T17:52:57.564 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar (31 kB at 1.5 MB/s) 2024-03-04T17:52:57.575 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom 2024-03-04T17:52:57.597 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 9.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom (9.8 kB at 446 kB/s) 2024-03-04T17:52:57.609 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar 2024-03-04T17:52:57.632 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/66 kB Progress (1): 33/66 kB Progress (1): 49/66 kB Progress (1): 66/66 kB Progress (1): 66 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar (66 kB at 2.9 MB/s) 2024-03-04T17:52:57.644 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom 2024-03-04T17:52:57.669 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom (5.3 kB at 213 kB/s) 2024-03-04T17:52:57.673 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom 2024-03-04T17:52:57.694 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom (22 kB at 1.0 MB/s) 2024-03-04T17:52:57.702 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom 2024-03-04T17:52:57.723 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/50 kB Progress (1): 33/50 kB Progress (1): 49/50 kB Progress (1): 50 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom (50 kB at 2.3 MB/s) 2024-03-04T17:52:57.733 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom 2024-03-04T17:52:57.754 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/24 kB Progress (1): 24 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom (24 kB at 1.1 MB/s) 2024-03-04T17:52:57.768 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom 2024-03-04T17:52:57.791 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom (5.6 kB at 245 kB/s) 2024-03-04T17:52:57.799 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar 2024-03-04T17:52:57.824 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/45 kB Progress (1): 33/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar (45 kB at 1.9 MB/s) 2024-03-04T17:52:57.836 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom 2024-03-04T17:52:57.856 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom (6.8 kB at 338 kB/s) 2024-03-04T17:52:57.862 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom 2024-03-04T17:52:57.881 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom (9.9 kB at 495 kB/s) 2024-03-04T17:52:57.886 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom 2024-03-04T17:52:57.909 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/46 kB Progress (1): 33/46 kB Progress (1): 46 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom (46 kB at 2.0 MB/s) 2024-03-04T17:52:57.916 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom 2024-03-04T17:52:57.936 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom (20 kB at 970 kB/s) 2024-03-04T17:52:57.946 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar 2024-03-04T17:52:57.972 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar (27 kB at 1.0 MB/s) 2024-03-04T17:52:58.005 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom 2024-03-04T17:52:58.025 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom (6.1 kB at 304 kB/s) 2024-03-04T17:52:58.030 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom 2024-03-04T17:52:58.057 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom (21 kB at 762 kB/s) 2024-03-04T17:52:58.064 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom 2024-03-04T17:52:58.083 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom (20 kB at 1.0 MB/s) 2024-03-04T17:52:58.094 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom 2024-03-04T17:52:58.113 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom (4.0 kB at 200 kB/s) 2024-03-04T17:52:58.116 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom 2024-03-04T17:52:58.135 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom (1.6 kB at 82 kB/s) 2024-03-04T17:52:58.137 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom 2024-03-04T17:52:58.156 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom (2.2 kB at 117 kB/s) 2024-03-04T17:52:58.161 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom 2024-03-04T17:52:58.179 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom (3.6 kB at 202 kB/s) 2024-03-04T17:52:58.185 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom 2024-03-04T17:52:58.208 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom (3.2 kB at 140 kB/s) 2024-03-04T17:52:58.213 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom 2024-03-04T17:52:58.232 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom (11 kB at 577 kB/s) 2024-03-04T17:52:58.237 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom 2024-03-04T17:52:58.255 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom (1.7 kB at 89 kB/s) 2024-03-04T17:52:58.260 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom 2024-03-04T17:52:58.278 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom (3.8 kB at 201 kB/s) 2024-03-04T17:52:58.283 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom 2024-03-04T17:52:58.301 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom (4.9 kB at 273 kB/s) 2024-03-04T17:52:58.306 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom 2024-03-04T17:52:58.324 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom (1.2 kB at 64 kB/s) 2024-03-04T17:52:58.327 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom 2024-03-04T17:52:58.344 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom (3.8 kB at 213 kB/s) 2024-03-04T17:52:58.347 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom 2024-03-04T17:52:58.365 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom (14 kB at 767 kB/s) 2024-03-04T17:52:58.372 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom 2024-03-04T17:52:58.394 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom (3.7 kB at 167 kB/s) 2024-03-04T17:52:58.406 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom 2024-03-04T17:52:58.428 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom (2.5 kB at 111 kB/s) 2024-03-04T17:52:58.435 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom 2024-03-04T17:52:58.454 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom (2.9 kB at 146 kB/s) 2024-03-04T17:52:58.464 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom 2024-03-04T17:52:58.490 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 7.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom (7.4 kB at 274 kB/s) 2024-03-04T17:52:58.502 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom 2024-03-04T17:52:58.522 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 5.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom (5.5 kB at 260 kB/s) 2024-03-04T17:52:58.532 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom 2024-03-04T17:52:58.553 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom (3.5 kB at 166 kB/s) 2024-03-04T17:52:58.565 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom 2024-03-04T17:52:58.588 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom (4.4 kB at 191 kB/s) 2024-03-04T17:52:58.593 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom 2024-03-04T17:52:58.615 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom (2.4 kB at 115 kB/s) 2024-03-04T17:52:58.622 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom 2024-03-04T17:52:58.642 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom (6.4 kB at 322 kB/s) 2024-03-04T17:52:58.648 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom 2024-03-04T17:52:58.675 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom (6.1 kB at 225 kB/s) 2024-03-04T17:52:58.683 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom 2024-03-04T17:52:58.703 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom (3.0 kB at 142 kB/s) 2024-03-04T17:52:58.711 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom 2024-03-04T17:52:58.731 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom (3.5 kB at 174 kB/s) 2024-03-04T17:52:58.740 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom 2024-03-04T17:52:58.759 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom (4.1 kB at 204 kB/s) 2024-03-04T17:52:58.763 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom 2024-03-04T17:52:58.783 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom (1.8 kB at 94 kB/s) 2024-03-04T17:52:58.792 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom 2024-03-04T17:52:58.812 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom (6.6 kB at 330 kB/s) 2024-03-04T17:52:58.816 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom 2024-03-04T17:52:58.835 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom (16 kB at 862 kB/s) 2024-03-04T17:52:58.841 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom 2024-03-04T17:52:58.866 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/35 kB Progress (1): 32/35 kB Progress (1): 35 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom (35 kB at 1.3 MB/s) 2024-03-04T17:52:58.874 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom 2024-03-04T17:52:58.894 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom (17 kB at 855 kB/s) 2024-03-04T17:52:58.903 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom 2024-03-04T17:52:58.923 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom (5.0 kB at 261 kB/s) 2024-03-04T17:52:58.928 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom 2024-03-04T17:52:58.947 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom (13 kB at 693 kB/s) 2024-03-04T17:52:58.957 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom 2024-03-04T17:52:58.978 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom (14 kB at 698 kB/s) 2024-03-04T17:52:58.983 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom 2024-03-04T17:52:59.005 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/68 kB Progress (1): 32/68 kB Progress (1): 49/68 kB Progress (1): 65/68 kB Progress (1): 68 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom (68 kB at 3.0 MB/s) 2024-03-04T17:52:59.021 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2024-03-04T17:52:59.041 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom (16 kB at 746 kB/s) 2024-03-04T17:52:59.052 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom 2024-03-04T17:52:59.071 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom (15 kB at 813 kB/s) 2024-03-04T17:52:59.076 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom 2024-03-04T17:52:59.099 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/79 kB Progress (1): 33/79 kB Progress (1): 49/79 kB Progress (1): 66/79 kB Progress (1): 79 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom (79 kB at 3.3 MB/s) 2024-03-04T17:52:59.110 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom 2024-03-04T17:52:59.129 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/18 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom (18 kB at 921 kB/s) 2024-03-04T17:52:59.138 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom 2024-03-04T17:52:59.159 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom (20 kB at 908 kB/s) 2024-03-04T17:52:59.166 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom 2024-03-04T17:52:59.190 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/78 kB Progress (1): 32/78 kB Progress (1): 49/78 kB Progress (1): 65/78 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom (78 kB at 3.2 MB/s) 2024-03-04T17:52:59.208 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom 2024-03-04T17:52:59.234 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom (3.8 kB at 148 kB/s) 2024-03-04T17:52:59.238 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom 2024-03-04T17:52:59.259 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom (14 kB at 657 kB/s) 2024-03-04T17:52:59.265 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom 2024-03-04T17:52:59.290 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom (3.2 kB at 128 kB/s) 2024-03-04T17:52:59.295 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom 2024-03-04T17:52:59.315 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom (3.2 kB at 159 kB/s) 2024-03-04T17:52:59.321 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom 2024-03-04T17:52:59.340 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom (2.0 kB at 107 kB/s) 2024-03-04T17:52:59.345 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom 2024-03-04T17:52:59.364 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom (2.8 kB at 142 kB/s) 2024-03-04T17:52:59.370 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom 2024-03-04T17:52:59.390 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom (1.5 kB at 76 kB/s) 2024-03-04T17:52:59.395 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom 2024-03-04T17:52:59.414 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom (3.0 kB at 158 kB/s) 2024-03-04T17:52:59.419 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom 2024-03-04T17:52:59.438 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom (3.2 kB at 160 kB/s) 2024-03-04T17:52:59.443 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom 2024-03-04T17:52:59.463 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom (3.2 kB at 160 kB/s) 2024-03-04T17:52:59.469 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom 2024-03-04T17:52:59.488 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom (1.0 kB at 51 kB/s) 2024-03-04T17:52:59.496 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom 2024-03-04T17:52:59.514 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 964 B Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom (964 B at 51 kB/s) 2024-03-04T17:52:59.520 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom 2024-03-04T17:52:59.539 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom (3.1 kB at 153 kB/s) 2024-03-04T17:52:59.544 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom 2024-03-04T17:52:59.563 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom (3.0 kB at 160 kB/s) 2024-03-04T17:52:59.568 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom 2024-03-04T17:52:59.586 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom (3.4 kB at 181 kB/s) 2024-03-04T17:52:59.591 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom 2024-03-04T17:52:59.611 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom (3.4 kB at 181 kB/s) 2024-03-04T17:52:59.615 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom 2024-03-04T17:52:59.636 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom (3.0 kB at 144 kB/s) 2024-03-04T17:52:59.687 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar 2024-03-04T17:52:59.847 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3 MB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar (3.3 MB at 20 MB/s) 2024-03-04T17:52:59.850 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar 2024-03-04T17:52:59.850 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar 2024-03-04T17:52:59.850 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar 2024-03-04T17:52:59.851 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar 2024-03-04T17:52:59.851 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar 2024-03-04T17:52:59.881 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/98 kB Progress (1): 33/98 kB Progress (1): 49/98 kB Progress (1): 66/98 kB Progress (1): 82/98 kB Progress (1): 98 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar (98 kB at 2.9 MB/s) 2024-03-04T17:52:59.881 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar 2024-03-04T17:52:59.903 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar (16 kB at 308 kB/s) 2024-03-04T17:52:59.903 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar 2024-03-04T17:52:59.923 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.4/16 kB Progress (1): 2.8/16 kB Progress (1): 4.1/16 kB Progress (1): 5.5/16 kB Progress (1): 6.9/16 kB Progress (1): 8.3/16 kB Progress (1): 9.7/16 kB Progress (1): 11/16 kB Progress (1): 12/16 kB Progress (1): 14/16 kB Progress (1): 15/16 kB Progress (1): 16 kB Progress (2): 16 kB | 16/782 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar (16 kB at 216 kB/s) 2024-03-04T17:52:59.923 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar 2024-03-04T17:52:59.943 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 33/782 kB Progress (1): 49/782 kB Progress (1): 66/782 kB Progress (1): 82/782 kB Progress (1): 98/782 kB Progress (1): 115/782 kB Progress (1): 131/782 kB Progress (1): 147/782 kB Progress (1): 164/782 kB Progress (1): 180/782 kB Progress (1): 197/782 kB Progress (1): 213/782 kB Progress (1): 229/782 kB Progress (1): 246/782 kB Progress (1): 256/782 kB Progress (1): 272/782 kB Progress (1): 289/782 kB Progress (1): 305/782 kB Progress (1): 321/782 kB Progress (1): 338/782 kB Progress (1): 354/782 kB Progress (1): 370/782 kB Progress (1): 387/782 kB Progress (1): 403/782 kB Progress (1): 420/782 kB Progress (1): 436/782 kB Progress (1): 452/782 kB Progress (1): 469/782 kB Progress (1): 485/782 kB Progress (1): 502/782 kB Progress (1): 518/782 kB Progress (1): 534/782 kB Progress (1): 551/782 kB Progress (1): 567/782 kB Progress (1): 583/782 kB Progress (1): 600/782 kB Progress (1): 616/782 kB Progress (1): 633/782 kB Progress (1): 649/782 kB Progress (1): 665/782 kB Progress (1): 682/782 kB Progress (1): 698/782 kB Progress (1): 715/782 kB Progress (1): 731/782 kB Progress (2): 731/782 kB | 1.4/71 kB Progress (2): 747/782 kB | 1.4/71 kB Progress (2): 747/782 kB | 2.8/71 kB Progress (2): 764/782 kB | 2.8/71 kB Progress (2): 764/782 kB | 4.1/71 kB Progress (2): 780/782 kB | 4.1/71 kB Progress (2): 780/782 kB | 5.5/71 kB Progress (2): 782 kB | 5.5/71 kB Progress (2): 782 kB | 6.9/71 kB Progress (3): 782 kB | 6.9/71 kB | 1.4/49 kB Progress (3): 782 kB | 8.3/71 kB | 1.4/49 kB Progress (3): 782 kB | 8.3/71 kB | 2.8/49 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar (782 kB at 8.5 MB/s) 2024-03-04T17:52:59.943 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar 2024-03-04T17:52:59.951 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (3): 8.3/71 kB | 2.8/49 kB | 1.4/35 kB Progress (3): 9.7/71 kB | 2.8/49 kB | 1.4/35 kB Progress (3): 9.7/71 kB | 4.1/49 kB | 1.4/35 kB Progress (3): 9.7/71 kB | 4.1/49 kB | 2.8/35 kB Progress (3): 9.7/71 kB | 5.5/49 kB | 2.8/35 kB Progress (3): 11/71 kB | 5.5/49 kB | 2.8/35 kB Progress (3): 11/71 kB | 5.5/49 kB | 4.1/35 kB Progress (3): 12/71 kB | 5.5/49 kB | 4.1/35 kB Progress (3): 12/71 kB | 6.9/49 kB | 4.1/35 kB Progress (3): 12/71 kB | 6.9/49 kB | 5.5/35 kB Progress (3): 14/71 kB | 6.9/49 kB | 5.5/35 kB Progress (3): 14/71 kB | 8.3/49 kB | 5.5/35 kB Progress (3): 15/71 kB | 8.3/49 kB | 5.5/35 kB Progress (3): 15/71 kB | 8.3/49 kB | 6.9/35 kB Progress (3): 16/71 kB | 8.3/49 kB | 6.9/35 kB Progress (3): 16/71 kB | 9.7/49 kB | 6.9/35 kB Progress (3): 16/71 kB | 9.7/49 kB | 8.3/35 kB Progress (3): 16/71 kB | 11/49 kB | 8.3/35 kB Progress (3): 32/71 kB | 11/49 kB | 8.3/35 kB Progress (3): 32/71 kB | 11/49 kB | 9.7/35 kB Progress (3): 32/71 kB | 12/49 kB | 9.7/35 kB Progress (3): 32/71 kB | 14/49 kB | 9.7/35 kB Progress (3): 32/71 kB | 14/49 kB | 11/35 kB Progress (3): 32/71 kB | 15/49 kB | 11/35 kB Progress (3): 32/71 kB | 15/49 kB | 12/35 kB Progress (3): 32/71 kB | 17/49 kB | 12/35 kB Progress (3): 32/71 kB | 17/49 kB | 14/35 kB Progress (3): 32/71 kB | 18/49 kB | 14/35 kB Progress (3): 32/71 kB | 18/49 kB | 15/35 kB Progress (3): 32/71 kB | 19/49 kB | 15/35 kB Progress (3): 32/71 kB | 19/49 kB | 17/35 kB Progress (3): 32/71 kB | 21/49 kB | 17/35 kB Progress (3): 32/71 kB | 22/49 kB | 17/35 kB Progress (3): 32/71 kB | 22/49 kB | 18/35 kB Progress (3): 32/71 kB | 23/49 kB | 18/35 kB Progress (3): 32/71 kB | 25/49 kB | 18/35 kB Progress (3): 32/71 kB | 25/49 kB | 19/35 kB Progress (3): 32/71 kB | 26/49 kB | 19/35 kB Progress (4): 32/71 kB | 26/49 kB | 19/35 kB | 1.4/11 kB Progress (4): 32/71 kB | 26/49 kB | 21/35 kB | 1.4/11 kB Progress (4): 32/71 kB | 28/49 kB | 21/35 kB | 1.4/11 kB Progress (4): 32/71 kB | 28/49 kB | 21/35 kB | 2.8/11 kB Progress (4): 32/71 kB | 28/49 kB | 22/35 kB | 2.8/11 kB Progress (4): 32/71 kB | 28/49 kB | 22/35 kB | 4.1/11 kB Progress (4): 32/71 kB | 29/49 kB | 22/35 kB | 4.1/11 kB Progress (4): 32/71 kB | 29/49 kB | 23/35 kB | 4.1/11 kB Progress (4): 32/71 kB | 30/49 kB | 23/35 kB | 4.1/11 kB Progress (4): 32/71 kB | 30/49 kB | 25/35 kB | 4.1/11 kB Progress (4): 32/71 kB | 30/49 kB | 25/35 kB | 5.5/11 kB Progress (4): 32/71 kB | 30/49 kB | 26/35 kB | 5.5/11 kB Progress (4): 32/71 kB | 30/49 kB | 26/35 kB | 6.9/11 kB Progress (4): 32/71 kB | 30/49 kB | 28/35 kB | 6.9/11 kB Progress (4): 32/71 kB | 30/49 kB | 28/35 kB | 8.3/11 kB Progress (4): 32/71 kB | 32/49 kB | 28/35 kB | 8.3/11 kB Progress (4): 32/71 kB | 32/49 kB | 28/35 kB | 9.7/11 kB Progress (4): 32/71 kB | 32/49 kB | 29/35 kB | 9.7/11 kB Progress (4): 32/71 kB | 32/49 kB | 29/35 kB | 11/11 kB Progress (4): 32/71 kB | 32/49 kB | 30/35 kB | 11/11 kB Progress (4): 32/71 kB | 33/49 kB | 30/35 kB | 11/11 kB Progress (4): 32/71 kB | 33/49 kB | 32/35 kB | 11/11 kB Progress (4): 32/71 kB | 34/49 kB | 32/35 kB | 11/11 kB Progress (4): 32/71 kB | 34/49 kB | 32/35 kB | 11 kB Progress (4): 32/71 kB | 34/49 kB | 33/35 kB | 11 kB Progress (4): 32/71 kB | 36/49 kB | 33/35 kB | 11 kB Progress (4): 32/71 kB | 36/49 kB | 34/35 kB | 11 kB Progress (4): 32/71 kB | 37/49 kB | 34/35 kB | 11 kB Progress (4): 32/71 kB | 37/49 kB | 35 kB | 11 kB Progress (4): 32/71 kB | 39/49 kB | 35 kB | 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar (11 kB at 115 kB/s) 2024-03-04T17:52:59.951 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar 2024-03-04T17:52:59.952 INFO:tasks.workunit.client.0.smithi137.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar (35 kB at 345 kB/s) 2024-03-04T17:52:59.952 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar 2024-03-04T17:52:59.960 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 49/71 kB | 39/49 kB Progress (2): 49/71 kB | 40/49 kB Progress (2): 49/71 kB | 41/49 kB Progress (2): 49/71 kB | 43/49 kB Progress (2): 49/71 kB | 44/49 kB Progress (2): 49/71 kB | 46/49 kB Progress (2): 49/71 kB | 47/49 kB Progress (2): 49/71 kB | 48/49 kB Progress (2): 49/71 kB | 49 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar (49 kB at 452 kB/s) 2024-03-04T17:52:59.960 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar 2024-03-04T17:52:59.962 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 65/71 kB Progress (2): 65/71 kB | 16/205 kB Progress (2): 71 kB | 16/205 kB Progress (2): 71 kB | 33/205 kB Progress (2): 71 kB | 49/205 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar (71 kB at 619 kB/s) 2024-03-04T17:52:59.963 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar 2024-03-04T17:52:59.967 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 66/205 kB Progress (1): 82/205 kB Progress (1): 98/205 kB Progress (1): 115/205 kB Progress (1): 131/205 kB Progress (1): 147/205 kB Progress (1): 164/205 kB Progress (1): 180/205 kB Progress (1): 197/205 kB Progress (1): 205 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar (205 kB at 1.8 MB/s) 2024-03-04T17:52:59.967 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar 2024-03-04T17:52:59.972 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.4/30 kB Progress (1): 2.8/30 kB Progress (1): 4.1/30 kB Progress (1): 5.5/30 kB Progress (1): 6.9/30 kB Progress (1): 8.3/30 kB Progress (1): 9.7/30 kB Progress (1): 11/30 kB Progress (1): 12/30 kB Progress (1): 14/30 kB Progress (1): 15/30 kB Progress (1): 16/30 kB Progress (1): 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar (30 kB at 250 kB/s) 2024-03-04T17:52:59.972 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar 2024-03-04T17:52:59.980 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/81 kB Progress (1): 33/81 kB Progress (2): 33/81 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar (13 kB at 103 kB/s) 2024-03-04T17:52:59.980 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar 2024-03-04T17:53:00.020 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 33/81 kB | 16/798 kB Progress (2): 33/81 kB | 33/798 kB Progress (2): 33/81 kB | 37/798 kB Progress (2): 33/81 kB | 53/798 kB Progress (2): 49/81 kB | 53/798 kB Progress (2): 49/81 kB | 69/798 kB Progress (2): 66/81 kB | 69/798 kB Progress (2): 81 kB | 69/798 kB Progress (2): 81 kB | 86/798 kB Progress (2): 81 kB | 102/798 kB Progress (2): 81 kB | 118/798 kB Progress (3): 81 kB | 118/798 kB | 16/182 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar (81 kB at 490 kB/s) 2024-03-04T17:53:00.020 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 135/798 kB | 16/182 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar 2024-03-04T17:53:00.025 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 135/798 kB | 33/182 kB Progress (2): 151/798 kB | 33/182 kB Progress (2): 168/798 kB | 33/182 kB Progress (2): 184/798 kB | 33/182 kB Progress (2): 200/798 kB | 33/182 kB Progress (2): 217/798 kB | 33/182 kB Progress (2): 233/798 kB | 33/182 kB Progress (2): 250/798 kB | 33/182 kB Progress (2): 266/798 kB | 33/182 kB Progress (2): 282/798 kB | 33/182 kB Progress (2): 299/798 kB | 33/182 kB Progress (2): 315/798 kB | 33/182 kB Progress (2): 331/798 kB | 33/182 kB Progress (3): 331/798 kB | 33/182 kB | 16/151 kB Progress (3): 331/798 kB | 49/182 kB | 16/151 kB Progress (4): 331/798 kB | 49/182 kB | 16/151 kB | 16/27 kB Progress (4): 348/798 kB | 49/182 kB | 16/151 kB | 16/27 kB Progress (4): 348/798 kB | 49/182 kB | 33/151 kB | 16/27 kB Progress (4): 348/798 kB | 66/182 kB | 33/151 kB | 16/27 kB Progress (4): 348/798 kB | 66/182 kB | 33/151 kB | 27 kB Progress (4): 364/798 kB | 66/182 kB | 33/151 kB | 27 kB Progress (4): 364/798 kB | 66/182 kB | 49/151 kB | 27 kB Progress (4): 364/798 kB | 82/182 kB | 49/151 kB | 27 kB Progress (4): 381/798 kB | 82/182 kB | 49/151 kB | 27 kB Progress (4): 381/798 kB | 98/182 kB | 49/151 kB | 27 kB Progress (4): 381/798 kB | 98/182 kB | 66/151 kB | 27 kB Progress (4): 397/798 kB | 98/182 kB | 66/151 kB | 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar (27 kB at 159 kB/s) 2024-03-04T17:53:00.026 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (3): 397/798 kB | 115/182 kB | 66/151 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar 2024-03-04T17:53:00.027 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (3): 397/798 kB | 115/182 kB | 82/151 kB Progress (3): 413/798 kB | 115/182 kB | 82/151 kB Progress (3): 413/798 kB | 131/182 kB | 82/151 kB Progress (3): 413/798 kB | 131/182 kB | 98/151 kB Progress (3): 413/798 kB | 147/182 kB | 98/151 kB Progress (3): 430/798 kB | 147/182 kB | 98/151 kB Progress (3): 430/798 kB | 147/182 kB | 115/151 kB Progress (3): 430/798 kB | 164/182 kB | 115/151 kB Progress (3): 446/798 kB | 164/182 kB | 115/151 kB Progress (3): 446/798 kB | 164/182 kB | 131/151 kB Progress (3): 446/798 kB | 180/182 kB | 131/151 kB Progress (3): 463/798 kB | 180/182 kB | 131/151 kB Progress (3): 463/798 kB | 182 kB | 131/151 kB Progress (3): 463/798 kB | 182 kB | 147/151 kB Progress (3): 463/798 kB | 182 kB | 151 kB Progress (3): 479/798 kB | 182 kB | 151 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar (182 kB at 1.0 MB/s) 2024-03-04T17:53:00.027 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 495/798 kB | 151 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar 2024-03-04T17:53:00.027 INFO:tasks.workunit.client.0.smithi137.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar (151 kB at 866 kB/s) 2024-03-04T17:53:00.027 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar 2024-03-04T17:53:00.047 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 512/798 kB Progress (1): 528/798 kB Progress (1): 544/798 kB Progress (1): 548/798 kB Progress (1): 565/798 kB Progress (1): 581/798 kB Progress (1): 597/798 kB Progress (1): 614/798 kB Progress (1): 630/798 kB Progress (1): 646/798 kB Progress (1): 663/798 kB Progress (1): 679/798 kB Progress (1): 696/798 kB Progress (1): 712/798 kB Progress (1): 728/798 kB Progress (1): 745/798 kB Progress (1): 761/798 kB Progress (1): 778/798 kB Progress (1): 794/798 kB Progress (1): 798 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar (798 kB at 4.1 MB/s) 2024-03-04T17:53:00.047 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar 2024-03-04T17:53:00.051 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/75 kB Progress (1): 32/75 kB Progress (1): 49/75 kB Progress (1): 65/75 kB Progress (2): 65/75 kB | 13 kB Progress (2): 75 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar (13 kB at 66 kB/s) 2024-03-04T17:53:00.051 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar 2024-03-04T17:53:00.052 INFO:tasks.workunit.client.0.smithi137.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar (75 kB at 376 kB/s) 2024-03-04T17:53:00.052 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar 2024-03-04T17:53:00.063 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/385 kB Progress (1): 32/385 kB Progress (1): 49/385 kB Progress (1): 65/385 kB Progress (2): 65/385 kB | 16/30 kB Progress (2): 81/385 kB | 16/30 kB Progress (2): 81/385 kB | 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar (30 kB at 143 kB/s) 2024-03-04T17:53:00.063 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar 2024-03-04T17:53:00.086 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 98/385 kB Progress (2): 98/385 kB | 16/780 kB Progress (2): 98/385 kB | 33/780 kB Progress (2): 98/385 kB | 49/780 kB Progress (2): 98/385 kB | 66/780 kB Progress (2): 98/385 kB | 82/780 kB Progress (2): 98/385 kB | 98/780 kB Progress (2): 98/385 kB | 115/780 kB Progress (2): 98/385 kB | 131/780 kB Progress (2): 98/385 kB | 147/780 kB Progress (2): 98/385 kB | 164/780 kB Progress (3): 98/385 kB | 164/780 kB | 16/329 kB Progress (3): 98/385 kB | 164/780 kB | 33/329 kB Progress (3): 98/385 kB | 164/780 kB | 49/329 kB Progress (3): 98/385 kB | 164/780 kB | 66/329 kB Progress (3): 98/385 kB | 164/780 kB | 82/329 kB Progress (3): 98/385 kB | 164/780 kB | 98/329 kB Progress (3): 98/385 kB | 180/780 kB | 98/329 kB Progress (3): 114/385 kB | 180/780 kB | 98/329 kB Progress (3): 114/385 kB | 180/780 kB | 115/329 kB Progress (3): 114/385 kB | 197/780 kB | 115/329 kB Progress (3): 114/385 kB | 197/780 kB | 131/329 kB Progress (3): 114/385 kB | 197/780 kB | 147/329 kB Progress (3): 114/385 kB | 197/780 kB | 164/329 kB Progress (3): 114/385 kB | 213/780 kB | 164/329 kB Progress (3): 114/385 kB | 213/780 kB | 180/329 kB Progress (3): 114/385 kB | 229/780 kB | 180/329 kB Progress (3): 114/385 kB | 229/780 kB | 197/329 kB Progress (3): 114/385 kB | 229/780 kB | 213/329 kB Progress (3): 114/385 kB | 246/780 kB | 213/329 kB Progress (3): 114/385 kB | 256/780 kB | 213/329 kB Progress (4): 114/385 kB | 256/780 kB | 213/329 kB | 16/501 kB Progress (4): 114/385 kB | 256/780 kB | 229/329 kB | 16/501 kB Progress (4): 131/385 kB | 256/780 kB | 229/329 kB | 16/501 kB Progress (4): 131/385 kB | 256/780 kB | 246/329 kB | 16/501 kB Progress (4): 131/385 kB | 256/780 kB | 256/329 kB | 16/501 kB Progress (4): 131/385 kB | 272/780 kB | 256/329 kB | 16/501 kB Progress (4): 131/385 kB | 272/780 kB | 256/329 kB | 33/501 kB Progress (4): 131/385 kB | 289/780 kB | 256/329 kB | 33/501 kB Progress (4): 131/385 kB | 289/780 kB | 272/329 kB | 33/501 kB Progress (4): 131/385 kB | 305/780 kB | 272/329 kB | 33/501 kB Progress (4): 131/385 kB | 305/780 kB | 272/329 kB | 49/501 kB Progress (4): 131/385 kB | 305/780 kB | 289/329 kB | 49/501 kB Progress (4): 131/385 kB | 321/780 kB | 289/329 kB | 49/501 kB Progress (4): 131/385 kB | 321/780 kB | 289/329 kB | 66/501 kB Progress (4): 131/385 kB | 321/780 kB | 305/329 kB | 66/501 kB Progress (4): 131/385 kB | 321/780 kB | 321/329 kB | 66/501 kB Progress (4): 147/385 kB | 321/780 kB | 321/329 kB | 66/501 kB Progress (4): 147/385 kB | 338/780 kB | 321/329 kB | 66/501 kB Progress (4): 147/385 kB | 338/780 kB | 329 kB | 66/501 kB Progress (4): 147/385 kB | 338/780 kB | 329 kB | 82/501 kB Progress (4): 147/385 kB | 354/780 kB | 329 kB | 82/501 kB Progress (4): 147/385 kB | 370/780 kB | 329 kB | 82/501 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar (329 kB at 1.4 MB/s) 2024-03-04T17:53:00.086 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (3): 147/385 kB | 387/780 kB | 82/501 kB Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar 2024-03-04T17:53:00.103 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (3): 147/385 kB | 387/780 kB | 98/501 kB Progress (3): 147/385 kB | 387/780 kB | 115/501 kB Progress (3): 147/385 kB | 403/780 kB | 115/501 kB Progress (3): 147/385 kB | 403/780 kB | 131/501 kB Progress (3): 147/385 kB | 420/780 kB | 131/501 kB Progress (3): 163/385 kB | 420/780 kB | 131/501 kB Progress (3): 163/385 kB | 436/780 kB | 131/501 kB Progress (3): 163/385 kB | 436/780 kB | 147/501 kB Progress (3): 163/385 kB | 452/780 kB | 147/501 kB Progress (3): 163/385 kB | 469/780 kB | 147/501 kB Progress (3): 163/385 kB | 469/780 kB | 164/501 kB Progress (3): 163/385 kB | 485/780 kB | 164/501 kB Progress (3): 163/385 kB | 485/780 kB | 180/501 kB Progress (3): 180/385 kB | 485/780 kB | 180/501 kB Progress (3): 180/385 kB | 502/780 kB | 180/501 kB Progress (3): 180/385 kB | 518/780 kB | 180/501 kB Progress (3): 180/385 kB | 518/780 kB | 197/501 kB Progress (3): 180/385 kB | 534/780 kB | 197/501 kB Progress (3): 180/385 kB | 534/780 kB | 213/501 kB Progress (3): 180/385 kB | 551/780 kB | 213/501 kB Progress (3): 196/385 kB | 551/780 kB | 213/501 kB Progress (3): 196/385 kB | 567/780 kB | 213/501 kB Progress (3): 196/385 kB | 567/780 kB | 229/501 kB Progress (3): 196/385 kB | 583/780 kB | 229/501 kB Progress (3): 196/385 kB | 600/780 kB | 229/501 kB Progress (4): 196/385 kB | 600/780 kB | 229/501 kB | 16/354 kB Progress (4): 196/385 kB | 616/780 kB | 229/501 kB | 16/354 kB Progress (4): 212/385 kB | 616/780 kB | 229/501 kB | 16/354 kB Progress (4): 212/385 kB | 616/780 kB | 229/501 kB | 33/354 kB Progress (4): 212/385 kB | 633/780 kB | 229/501 kB | 33/354 kB Progress (4): 229/385 kB | 633/780 kB | 229/501 kB | 33/354 kB Progress (4): 229/385 kB | 649/780 kB | 229/501 kB | 33/354 kB Progress (4): 229/385 kB | 665/780 kB | 229/501 kB | 33/354 kB Progress (4): 229/385 kB | 682/780 kB | 229/501 kB | 33/354 kB Progress (4): 229/385 kB | 682/780 kB | 229/501 kB | 49/354 kB Progress (4): 229/385 kB | 698/780 kB | 229/501 kB | 49/354 kB Progress (4): 229/385 kB | 715/780 kB | 229/501 kB | 49/354 kB Progress (4): 229/385 kB | 731/780 kB | 229/501 kB | 49/354 kB Progress (4): 229/385 kB | 747/780 kB | 229/501 kB | 49/354 kB Progress (4): 229/385 kB | 747/780 kB | 229/501 kB | 66/354 kB Progress (4): 229/385 kB | 764/780 kB | 229/501 kB | 66/354 kB Progress (4): 229/385 kB | 780/780 kB | 229/501 kB | 66/354 kB Progress (4): 229/385 kB | 780 kB | 229/501 kB | 66/354 kB Progress (4): 229/385 kB | 780 kB | 229/501 kB | 82/354 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar (780 kB at 3.1 MB/s) 2024-03-04T17:53:00.103 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar 2024-03-04T17:53:00.116 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (3): 229/385 kB | 229/501 kB | 98/354 kB Progress (3): 229/385 kB | 229/501 kB | 115/354 kB Progress (3): 245/385 kB | 229/501 kB | 115/354 kB Progress (3): 245/385 kB | 246/501 kB | 115/354 kB Progress (3): 245/385 kB | 256/501 kB | 115/354 kB Progress (3): 245/385 kB | 272/501 kB | 115/354 kB Progress (3): 262/385 kB | 272/501 kB | 115/354 kB Progress (3): 262/385 kB | 289/501 kB | 115/354 kB Progress (3): 262/385 kB | 305/501 kB | 115/354 kB Progress (3): 262/385 kB | 321/501 kB | 115/354 kB Progress (3): 278/385 kB | 321/501 kB | 115/354 kB Progress (3): 278/385 kB | 338/501 kB | 115/354 kB Progress (3): 278/385 kB | 354/501 kB | 115/354 kB Progress (3): 278/385 kB | 370/501 kB | 115/354 kB Progress (4): 278/385 kB | 370/501 kB | 115/354 kB | 16/41 kB Progress (4): 278/385 kB | 387/501 kB | 115/354 kB | 16/41 kB Progress (4): 294/385 kB | 387/501 kB | 115/354 kB | 16/41 kB Progress (4): 294/385 kB | 387/501 kB | 115/354 kB | 33/41 kB Progress (4): 294/385 kB | 403/501 kB | 115/354 kB | 33/41 kB Progress (4): 294/385 kB | 403/501 kB | 115/354 kB | 41 kB Progress (4): 294/385 kB | 420/501 kB | 115/354 kB | 41 kB Progress (4): 311/385 kB | 420/501 kB | 115/354 kB | 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar (41 kB at 156 kB/s) 2024-03-04T17:53:00.116 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar 2024-03-04T17:53:00.128 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (3): 311/385 kB | 420/501 kB | 131/354 kB Progress (3): 311/385 kB | 436/501 kB | 131/354 kB Progress (3): 327/385 kB | 436/501 kB | 131/354 kB Progress (3): 327/385 kB | 436/501 kB | 147/354 kB Progress (3): 327/385 kB | 452/501 kB | 147/354 kB Progress (3): 327/385 kB | 469/501 kB | 147/354 kB Progress (3): 343/385 kB | 469/501 kB | 147/354 kB Progress (3): 343/385 kB | 485/501 kB | 147/354 kB Progress (3): 343/385 kB | 485/501 kB | 164/354 kB Progress (3): 343/385 kB | 485/501 kB | 180/354 kB Progress (3): 360/385 kB | 485/501 kB | 180/354 kB Progress (3): 360/385 kB | 485/501 kB | 197/354 kB Progress (4): 360/385 kB | 485/501 kB | 197/354 kB | 6.4 kB Progress (4): 360/385 kB | 501 kB | 197/354 kB | 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar (6.4 kB at 23 kB/s) 2024-03-04T17:53:00.128 INFO:tasks.workunit.client.0.smithi137.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar (501 kB at 1.8 MB/s) 2024-03-04T17:53:00.128 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 376/385 kB | 197/354 kB Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar 2024-03-04T17:53:00.128 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar 2024-03-04T17:53:00.130 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 385 kB | 197/354 kB Progress (2): 385 kB | 213/354 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar (385 kB at 1.4 MB/s) 2024-03-04T17:53:00.130 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar 2024-03-04T17:53:00.155 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 229/354 kB Progress (1): 246/354 kB Progress (1): 256/354 kB Progress (1): 272/354 kB Progress (2): 272/354 kB | 16/586 kB Progress (2): 272/354 kB | 32/586 kB Progress (2): 272/354 kB | 49/586 kB Progress (2): 272/354 kB | 65/586 kB Progress (2): 272/354 kB | 81/586 kB Progress (2): 272/354 kB | 98/586 kB Progress (2): 289/354 kB | 98/586 kB Progress (2): 289/354 kB | 114/586 kB Progress (2): 289/354 kB | 130/586 kB Progress (2): 289/354 kB | 147/586 kB Progress (2): 289/354 kB | 163/586 kB Progress (2): 289/354 kB | 180/586 kB Progress (2): 305/354 kB | 180/586 kB Progress (2): 305/354 kB | 196/586 kB Progress (2): 305/354 kB | 212/586 kB Progress (2): 305/354 kB | 229/586 kB Progress (2): 305/354 kB | 245/586 kB Progress (2): 305/354 kB | 262/586 kB Progress (2): 305/354 kB | 278/586 kB Progress (2): 305/354 kB | 294/586 kB Progress (2): 321/354 kB | 294/586 kB Progress (2): 321/354 kB | 311/586 kB Progress (2): 321/354 kB | 327/586 kB Progress (2): 321/354 kB | 343/586 kB Progress (2): 321/354 kB | 360/586 kB Progress (2): 321/354 kB | 376/586 kB Progress (2): 321/354 kB | 393/586 kB Progress (2): 338/354 kB | 393/586 kB Progress (2): 338/354 kB | 402/586 kB Progress (2): 338/354 kB | 418/586 kB Progress (2): 338/354 kB | 435/586 kB Progress (2): 338/354 kB | 451/586 kB Progress (2): 338/354 kB | 468/586 kB Progress (2): 354 kB | 468/586 kB Progress (3): 354 kB | 468/586 kB | 14 kB Progress (3): 354 kB | 484/586 kB | 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar (14 kB at 48 kB/s) 2024-03-04T17:53:00.155 INFO:tasks.workunit.client.0.smithi137.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar (354 kB at 1.2 MB/s) 2024-03-04T17:53:00.155 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar 2024-03-04T17:53:00.155 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 500/586 kB Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar 2024-03-04T17:53:00.158 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 517/586 kB Progress (1): 533/586 kB Progress (1): 549/586 kB Progress (1): 566/586 kB Progress (1): 582/586 kB Progress (1): 586 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar (586 kB at 1.9 MB/s) 2024-03-04T17:53:00.158 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar 2024-03-04T17:53:00.169 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/106 kB Progress (1): 32/106 kB Progress (1): 49/106 kB Progress (2): 49/106 kB | 16/211 kB Progress (2): 49/106 kB | 32/211 kB Progress (2): 65/106 kB | 32/211 kB Progress (2): 65/106 kB | 49/211 kB Progress (2): 65/106 kB | 65/211 kB Progress (2): 81/106 kB | 65/211 kB Progress (2): 81/106 kB | 81/211 kB Progress (2): 81/106 kB | 98/211 kB Progress (2): 98/106 kB | 98/211 kB Progress (2): 98/106 kB | 114/211 kB Progress (2): 98/106 kB | 130/211 kB Progress (2): 106 kB | 130/211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar (106 kB at 338 kB/s) 2024-03-04T17:53:00.169 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar 2024-03-04T17:53:00.175 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 147/211 kB Progress (1): 163/211 kB Progress (1): 180/211 kB Progress (1): 196/211 kB Progress (1): 211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar (211 kB at 659 kB/s) 2024-03-04T17:53:00.175 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar 2024-03-04T17:53:00.176 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.8 kB Progress (2): 6.8 kB | 16/245 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar (6.8 kB at 21 kB/s) 2024-03-04T17:53:00.177 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar 2024-03-04T17:53:00.189 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 33/245 kB Progress (2): 33/245 kB | 16/205 kB Progress (2): 33/245 kB | 32/205 kB Progress (2): 33/245 kB | 49/205 kB Progress (2): 33/245 kB | 65/205 kB Progress (2): 33/245 kB | 81/205 kB Progress (2): 33/245 kB | 98/205 kB Progress (2): 33/245 kB | 114/205 kB Progress (2): 33/245 kB | 130/205 kB Progress (2): 33/245 kB | 147/205 kB Progress (2): 49/245 kB | 147/205 kB Progress (2): 49/245 kB | 163/205 kB Progress (2): 64/245 kB | 163/205 kB Progress (2): 64/245 kB | 180/205 kB Progress (2): 81/245 kB | 180/205 kB Progress (2): 81/245 kB | 196/205 kB Progress (2): 81/245 kB | 205 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar (205 kB at 613 kB/s) 2024-03-04T17:53:00.189 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar 2024-03-04T17:53:00.193 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 97/245 kB Progress (1): 113/245 kB Progress (2): 113/245 kB | 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar (15 kB at 45 kB/s) 2024-03-04T17:53:00.193 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar 2024-03-04T17:53:00.200 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 130/245 kB Progress (1): 146/245 kB Progress (1): 163/245 kB Progress (2): 163/245 kB | 6.4 kB Progress (2): 179/245 kB | 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar (6.4 kB at 18 kB/s) 2024-03-04T17:53:00.200 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar 2024-03-04T17:53:00.206 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 179/245 kB | 16/16 kB Progress (2): 179/245 kB | 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar (16 kB at 47 kB/s) 2024-03-04T17:53:00.206 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar 2024-03-04T17:53:00.214 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 195/245 kB Progress (1): 212/245 kB Progress (1): 228/245 kB Progress (2): 228/245 kB | 16/23 kB Progress (2): 228/245 kB | 23 kB Progress (2): 244/245 kB | 23 kB Progress (2): 245 kB | 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar (245 kB at 680 kB/s) 2024-03-04T17:53:00.214 INFO:tasks.workunit.client.0.smithi137.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar (23 kB at 63 kB/s) 2024-03-04T17:53:00.218 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/24 kB Progress (1): 24 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar (24 kB at 67 kB/s) 2024-03-04T17:53:00.232 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/184 kB Progress (2): 16/184 kB | 16/17 kB Progress (2): 16/184 kB | 17 kB Progress (2): 32/184 kB | 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar (17 kB at 46 kB/s) 2024-03-04T17:53:00.242 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 49/184 kB Progress (1): 65/184 kB Progress (1): 81/184 kB Progress (1): 98/184 kB Progress (1): 114/184 kB Progress (1): 130/184 kB Progress (1): 147/184 kB Progress (1): 163/184 kB Progress (1): 180/184 kB Progress (1): 184 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar (184 kB at 476 kB/s) 2024-03-04T17:53:00.265 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:53:00.265 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] --- clean:3.2.0:clean (default-clean) @ jcksum --- 2024-03-04T17:53:00.273 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom 2024-03-04T17:53:00.298 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom (5.8 kB at 243 kB/s) 2024-03-04T17:53:00.300 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom 2024-03-04T17:53:00.322 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom (5.1 kB at 232 kB/s) 2024-03-04T17:53:00.325 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom 2024-03-04T17:53:00.349 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/43 kB Progress (1): 33/43 kB Progress (1): 43 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom (43 kB at 1.7 MB/s) 2024-03-04T17:53:00.355 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom 2024-03-04T17:53:00.380 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom (14 kB at 570 kB/s) 2024-03-04T17:53:00.386 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar 2024-03-04T17:53:00.421 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/153 kB Progress (1): 32/153 kB Progress (1): 49/153 kB Progress (1): 65/153 kB Progress (1): 81/153 kB Progress (1): 98/153 kB Progress (1): 114/153 kB Progress (1): 130/153 kB Progress (1): 147/153 kB Progress (1): 153 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar (153 kB at 4.3 MB/s) 2024-03-04T17:53:00.422 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar 2024-03-04T17:53:00.469 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/215 kB Progress (1): 33/215 kB Progress (1): 49/215 kB Progress (1): 66/215 kB Progress (1): 82/215 kB Progress (1): 98/215 kB Progress (1): 115/215 kB Progress (1): 131/215 kB Progress (1): 147/215 kB Progress (1): 164/215 kB Progress (1): 180/215 kB Progress (1): 197/215 kB Progress (1): 213/215 kB Progress (1): 215 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar (215 kB at 4.5 MB/s) 2024-03-04T17:53:00.570 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:53:00.570 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2024-03-04T17:53:00.577 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom 2024-03-04T17:53:00.601 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom (2.7 kB at 106 kB/s) 2024-03-04T17:53:00.605 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom 2024-03-04T17:53:00.634 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom (23 kB at 776 kB/s) 2024-03-04T17:53:00.640 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom 2024-03-04T17:53:00.665 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 8.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom (8.8 kB at 351 kB/s) 2024-03-04T17:53:00.669 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom 2024-03-04T17:53:00.699 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom (25 kB at 847 kB/s) 2024-03-04T17:53:00.705 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom 2024-03-04T17:53:00.731 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom (6.0 kB at 224 kB/s) 2024-03-04T17:53:00.735 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom 2024-03-04T17:53:00.760 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom (3.2 kB at 129 kB/s) 2024-03-04T17:53:00.769 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom 2024-03-04T17:53:00.796 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 612 B Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom (612 B at 23 kB/s) 2024-03-04T17:53:00.800 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom 2024-03-04T17:53:00.826 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom (2.7 kB at 102 kB/s) 2024-03-04T17:53:00.831 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom 2024-03-04T17:53:00.858 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom (14 kB at 503 kB/s) 2024-03-04T17:53:00.864 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom 2024-03-04T17:53:00.891 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom (3.2 kB at 123 kB/s) 2024-03-04T17:53:00.895 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom 2024-03-04T17:53:00.922 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom (8.4 kB at 298 kB/s) 2024-03-04T17:53:00.926 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom 2024-03-04T17:53:00.953 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom (8.4 kB at 310 kB/s) 2024-03-04T17:53:00.962 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom 2024-03-04T17:53:00.987 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 8.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom (8.0 kB at 308 kB/s) 2024-03-04T17:53:00.995 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom 2024-03-04T17:53:01.025 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom (20 kB at 658 kB/s) 2024-03-04T17:53:01.040 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom 2024-03-04T17:53:01.068 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom (5.1 kB at 182 kB/s) 2024-03-04T17:53:01.074 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom 2024-03-04T17:53:01.101 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom (31 kB at 1.2 MB/s) 2024-03-04T17:53:01.115 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom 2024-03-04T17:53:01.139 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom (5.1 kB at 212 kB/s) 2024-03-04T17:53:01.151 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar 2024-03-04T17:53:01.189 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/85 kB Progress (1): 33/85 kB Progress (1): 49/85 kB Progress (1): 66/85 kB Progress (1): 82/85 kB Progress (1): 85 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar (85 kB at 2.3 MB/s) 2024-03-04T17:53:01.189 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar 2024-03-04T17:53:01.189 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar 2024-03-04T17:53:01.190 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar 2024-03-04T17:53:01.190 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar 2024-03-04T17:53:01.190 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar 2024-03-04T17:53:01.216 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 8.5 kB Progress (2): 8.5 kB | 2.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar (8.5 kB at 314 kB/s) 2024-03-04T17:53:01.217 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar 2024-03-04T17:53:01.217 INFO:tasks.workunit.client.0.smithi137.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar (2.5 kB at 92 kB/s) 2024-03-04T17:53:01.217 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar 2024-03-04T17:53:01.224 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/269 kB Progress (2): 16/269 kB | 16/55 kB Progress (2): 33/269 kB | 16/55 kB Progress (2): 49/269 kB | 16/55 kB Progress (2): 49/269 kB | 33/55 kB Progress (2): 66/269 kB | 33/55 kB Progress (2): 82/269 kB | 33/55 kB Progress (2): 82/269 kB | 49/55 kB Progress (2): 82/269 kB | 55 kB Progress (2): 98/269 kB | 55 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar (55 kB at 1.6 MB/s) 2024-03-04T17:53:01.238 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 115/269 kB Progress (1): 131/269 kB Progress (1): 147/269 kB Progress (1): 164/269 kB Progress (1): 180/269 kB Progress (1): 197/269 kB Progress (1): 213/269 kB Progress (2): 213/269 kB | 16/41 kB Progress (2): 229/269 kB | 16/41 kB Progress (2): 229/269 kB | 32/41 kB Progress (2): 246/269 kB | 32/41 kB Progress (2): 246/269 kB | 41 kB Progress (2): 256/269 kB | 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar (41 kB at 839 kB/s) 2024-03-04T17:53:01.244 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 269 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar (269 kB at 4.8 MB/s) 2024-03-04T17:53:01.262 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/587 kB Progress (1): 33/587 kB Progress (2): 33/587 kB | 16/327 kB Progress (2): 49/587 kB | 16/327 kB Progress (2): 49/587 kB | 32/327 kB Progress (2): 66/587 kB | 32/327 kB Progress (2): 66/587 kB | 49/327 kB Progress (2): 82/587 kB | 49/327 kB Progress (2): 98/587 kB | 49/327 kB Progress (2): 115/587 kB | 49/327 kB Progress (2): 115/587 kB | 65/327 kB Progress (2): 131/587 kB | 65/327 kB Progress (2): 131/587 kB | 81/327 kB Progress (2): 147/587 kB | 81/327 kB Progress (2): 164/587 kB | 81/327 kB Progress (2): 180/587 kB | 81/327 kB Progress (2): 180/587 kB | 98/327 kB Progress (2): 197/587 kB | 98/327 kB Progress (2): 197/587 kB | 114/327 kB Progress (2): 213/587 kB | 114/327 kB Progress (2): 229/587 kB | 114/327 kB Progress (2): 229/587 kB | 130/327 kB Progress (2): 246/587 kB | 130/327 kB Progress (2): 256/587 kB | 130/327 kB Progress (2): 256/587 kB | 147/327 kB Progress (2): 272/587 kB | 147/327 kB Progress (2): 289/587 kB | 147/327 kB Progress (2): 305/587 kB | 147/327 kB Progress (2): 305/587 kB | 163/327 kB Progress (2): 321/587 kB | 163/327 kB Progress (2): 321/587 kB | 180/327 kB Progress (2): 338/587 kB | 180/327 kB Progress (2): 354/587 kB | 180/327 kB Progress (2): 370/587 kB | 180/327 kB Progress (2): 387/587 kB | 180/327 kB Progress (2): 387/587 kB | 196/327 kB Progress (2): 387/587 kB | 212/327 kB Progress (2): 403/587 kB | 212/327 kB Progress (2): 420/587 kB | 212/327 kB Progress (2): 436/587 kB | 212/327 kB Progress (2): 436/587 kB | 229/327 kB Progress (2): 452/587 kB | 229/327 kB Progress (2): 452/587 kB | 245/327 kB Progress (2): 469/587 kB | 245/327 kB Progress (2): 485/587 kB | 245/327 kB Progress (2): 485/587 kB | 262/327 kB Progress (2): 485/587 kB | 278/327 kB Progress (2): 502/587 kB | 278/327 kB Progress (2): 518/587 kB | 278/327 kB Progress (2): 518/587 kB | 294/327 kB Progress (2): 534/587 kB | 294/327 kB Progress (2): 534/587 kB | 311/327 kB Progress (2): 551/587 kB | 311/327 kB Progress (2): 567/587 kB | 311/327 kB Progress (2): 567/587 kB | 327/327 kB Progress (2): 567/587 kB | 327 kB Progress (2): 583/587 kB | 327 kB Progress (2): 587 kB | 327 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar (327 kB at 4.6 MB/s) 2024-03-04T17:53:01.262 INFO:tasks.workunit.client.0.smithi137.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar (587 kB at 8.3 MB/s) 2024-03-04T17:53:01.314 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2024-03-04T17:53:01.315 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:53:01.315 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2024-03-04T17:53:01.319 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom 2024-03-04T17:53:01.346 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom (4.7 kB at 169 kB/s) 2024-03-04T17:53:01.348 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom 2024-03-04T17:53:01.374 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom (6.4 kB at 236 kB/s) 2024-03-04T17:53:01.376 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom 2024-03-04T17:53:01.405 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/33 kB Progress (1): 32/33 kB Progress (1): 33 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom (33 kB at 1.2 MB/s) 2024-03-04T17:53:01.407 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2024-03-04T17:53:01.438 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom (14 kB at 451 kB/s) 2024-03-04T17:53:01.441 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom 2024-03-04T17:53:01.473 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 815 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom (815 B at 25 kB/s) 2024-03-04T17:53:01.476 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom 2024-03-04T17:53:01.517 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom (4.2 kB at 101 kB/s) 2024-03-04T17:53:01.521 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom 2024-03-04T17:53:01.556 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom (17 kB at 494 kB/s) 2024-03-04T17:53:01.562 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom 2024-03-04T17:53:01.595 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom (5.0 kB at 146 kB/s) 2024-03-04T17:53:01.599 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom 2024-03-04T17:53:01.635 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom (4.1 kB at 112 kB/s) 2024-03-04T17:53:01.641 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom 2024-03-04T17:53:01.674 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom (2.4 kB at 72 kB/s) 2024-03-04T17:53:01.678 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom 2024-03-04T17:53:01.711 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom (11 kB at 332 kB/s) 2024-03-04T17:53:01.716 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom 2024-03-04T17:53:01.758 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom (17 kB at 410 kB/s) 2024-03-04T17:53:01.763 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2024-03-04T17:53:01.806 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom (6.6 kB at 156 kB/s) 2024-03-04T17:53:01.811 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom 2024-03-04T17:53:01.849 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom (1.1 kB at 28 kB/s) 2024-03-04T17:53:01.852 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom 2024-03-04T17:53:01.888 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom (8.4 kB at 233 kB/s) 2024-03-04T17:53:01.892 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom 2024-03-04T17:53:01.949 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom (2.7 kB at 46 kB/s) 2024-03-04T17:53:01.955 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom 2024-03-04T17:53:01.991 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom (5.6 kB at 156 kB/s) 2024-03-04T17:53:01.996 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom 2024-03-04T17:53:02.032 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom (1.1 kB at 32 kB/s) 2024-03-04T17:53:02.039 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom 2024-03-04T17:53:02.083 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 770 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom (770 B at 18 kB/s) 2024-03-04T17:53:02.086 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom 2024-03-04T17:53:02.123 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom (6.0 kB at 159 kB/s) 2024-03-04T17:53:02.127 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom 2024-03-04T17:53:02.161 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/26 kB Progress (1): 26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom (26 kB at 756 kB/s) 2024-03-04T17:53:02.167 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom 2024-03-04T17:53:02.204 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom (1.2 kB at 33 kB/s) 2024-03-04T17:53:02.208 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom 2024-03-04T17:53:02.243 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom (1.3 kB at 38 kB/s) 2024-03-04T17:53:02.256 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar 2024-03-04T17:53:02.290 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar (14 kB at 398 kB/s) 2024-03-04T17:53:02.290 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar 2024-03-04T17:53:02.291 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar 2024-03-04T17:53:02.291 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar 2024-03-04T17:53:02.291 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar 2024-03-04T17:53:02.292 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar 2024-03-04T17:53:02.319 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.2 kB Progress (2): 4.2 kB | 16/27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar (4.2 kB at 145 kB/s) 2024-03-04T17:53:02.319 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar 2024-03-04T17:53:02.320 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 16/27 kB | 16/55 kB Progress (2): 27 kB | 16/55 kB Progress (3): 27 kB | 16/55 kB | 16/334 kB Progress (3): 27 kB | 32/55 kB | 16/334 kB Progress (4): 27 kB | 32/55 kB | 16/334 kB | 16/122 kB Progress (4): 27 kB | 49/55 kB | 16/334 kB | 16/122 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar (27 kB at 965 kB/s) 2024-03-04T17:53:02.320 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (3): 49/55 kB | 16/334 kB | 33/122 kB Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar 2024-03-04T17:53:02.321 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (3): 55 kB | 16/334 kB | 33/122 kB Progress (3): 55 kB | 32/334 kB | 33/122 kB Progress (3): 55 kB | 32/334 kB | 49/122 kB Progress (3): 55 kB | 32/334 kB | 66/122 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar (55 kB at 1.8 MB/s) 2024-03-04T17:53:02.321 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar 2024-03-04T17:53:02.323 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 49/334 kB | 66/122 kB Progress (2): 49/334 kB | 82/122 kB Progress (2): 65/334 kB | 82/122 kB Progress (2): 65/334 kB | 98/122 kB Progress (2): 65/334 kB | 115/122 kB Progress (2): 65/334 kB | 122 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar (122 kB at 3.8 MB/s) 2024-03-04T17:53:02.344 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 81/334 kB Progress (1): 98/334 kB Progress (1): 114/334 kB Progress (1): 131/334 kB Progress (1): 147/334 kB Progress (1): 163/334 kB Progress (2): 163/334 kB | 4.7 kB Progress (3): 163/334 kB | 4.7 kB | 16/23 kB Progress (3): 180/334 kB | 4.7 kB | 16/23 kB Progress (4): 180/334 kB | 4.7 kB | 16/23 kB | 16/267 kB Progress (4): 180/334 kB | 4.7 kB | 23 kB | 16/267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar (4.7 kB at 88 kB/s) 2024-03-04T17:53:02.345 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (3): 196/334 kB | 23 kB | 16/267 kB Progress (3): 196/334 kB | 23 kB | 33/267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar (23 kB at 430 kB/s) 2024-03-04T17:53:02.369 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 196/334 kB | 49/267 kB Progress (2): 212/334 kB | 49/267 kB Progress (2): 212/334 kB | 66/267 kB Progress (2): 212/334 kB | 82/267 kB Progress (2): 212/334 kB | 98/267 kB Progress (2): 229/334 kB | 98/267 kB Progress (2): 229/334 kB | 115/267 kB Progress (2): 245/334 kB | 115/267 kB Progress (2): 245/334 kB | 131/267 kB Progress (2): 262/334 kB | 131/267 kB Progress (2): 262/334 kB | 147/267 kB Progress (2): 278/334 kB | 147/267 kB Progress (2): 278/334 kB | 164/267 kB Progress (2): 294/334 kB | 164/267 kB Progress (2): 294/334 kB | 180/267 kB Progress (2): 311/334 kB | 180/267 kB Progress (2): 311/334 kB | 197/267 kB Progress (2): 327/334 kB | 197/267 kB Progress (2): 334 kB | 197/267 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar (334 kB at 4.3 MB/s) 2024-03-04T17:53:02.389 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 213/267 kB Progress (1): 229/267 kB Progress (1): 246/267 kB Progress (1): 256/267 kB Progress (1): 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar (267 kB at 2.8 MB/s) 2024-03-04T17:53:02.751 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Changes detected - recompiling the module! :source 2024-03-04T17:53:02.754 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/classes 2024-03-04T17:53:03.980 INFO:tasks.workunit.client.0.smithi137.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2024-03-04T17:53:03.980 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:53:03.981 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2024-03-04T17:53:03.985 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2024-03-04T17:53:03.985 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:53:03.985 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2024-03-04T17:53:03.992 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Changes detected - recompiling the module! :dependency 2024-03-04T17:53:03.992 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/test-classes 2024-03-04T17:53:04.210 INFO:tasks.workunit.client.0.smithi137.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2024-03-04T17:53:04.211 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:53:04.211 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2024-03-04T17:53:04.217 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom 2024-03-04T17:53:04.255 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom (6.2 kB at 160 kB/s) 2024-03-04T17:53:04.261 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom 2024-03-04T17:53:04.300 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom (3.5 kB at 91 kB/s) 2024-03-04T17:53:04.304 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom 2024-03-04T17:53:04.343 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom (3.3 kB at 84 kB/s) 2024-03-04T17:53:04.349 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom 2024-03-04T17:53:04.389 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom (4.1 kB at 99 kB/s) 2024-03-04T17:53:04.395 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom 2024-03-04T17:53:04.434 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom (3.3 kB at 85 kB/s) 2024-03-04T17:53:04.439 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom 2024-03-04T17:53:04.479 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom (4.5 kB at 111 kB/s) 2024-03-04T17:53:04.484 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom 2024-03-04T17:53:04.525 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom (1.8 kB at 42 kB/s) 2024-03-04T17:53:04.530 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom 2024-03-04T17:53:04.572 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom (2.2 kB at 52 kB/s) 2024-03-04T17:53:04.574 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom 2024-03-04T17:53:04.617 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/30 kB Progress (1): 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom (30 kB at 701 kB/s) 2024-03-04T17:53:04.621 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom 2024-03-04T17:53:04.660 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom (1.9 kB at 49 kB/s) 2024-03-04T17:53:04.664 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom 2024-03-04T17:53:04.703 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom (5.8 kB at 149 kB/s) 2024-03-04T17:53:04.706 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom 2024-03-04T17:53:04.744 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom (2.3 kB at 60 kB/s) 2024-03-04T17:53:04.746 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom 2024-03-04T17:53:04.794 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom (22 kB at 466 kB/s) 2024-03-04T17:53:04.797 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom 2024-03-04T17:53:04.843 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/37 kB Progress (1): 33/37 kB Progress (1): 37 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom (37 kB at 832 kB/s) 2024-03-04T17:53:04.848 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2024-03-04T17:53:04.890 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom (15 kB at 354 kB/s) 2024-03-04T17:53:04.897 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom 2024-03-04T17:53:04.934 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 8.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom (8.7 kB at 228 kB/s) 2024-03-04T17:53:04.939 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom 2024-03-04T17:53:04.982 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom (27 kB at 622 kB/s) 2024-03-04T17:53:04.989 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom 2024-03-04T17:53:05.029 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom (8.1 kB at 202 kB/s) 2024-03-04T17:53:05.036 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom 2024-03-04T17:53:05.064 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom (2.2 kB at 75 kB/s) 2024-03-04T17:53:05.071 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom 2024-03-04T17:53:05.093 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom (2.6 kB at 113 kB/s) 2024-03-04T17:53:05.100 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom 2024-03-04T17:53:05.121 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom (1.5 kB at 77 kB/s) 2024-03-04T17:53:05.124 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom 2024-03-04T17:53:05.145 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom (3.1 kB at 146 kB/s) 2024-03-04T17:53:05.148 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom 2024-03-04T17:53:05.172 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom (20 kB at 889 kB/s) 2024-03-04T17:53:05.176 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom 2024-03-04T17:53:05.197 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom (6.8 kB at 307 kB/s) 2024-03-04T17:53:05.201 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom 2024-03-04T17:53:05.224 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom (14 kB at 590 kB/s) 2024-03-04T17:53:05.230 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom 2024-03-04T17:53:05.257 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom (3.0 kB at 110 kB/s) 2024-03-04T17:53:05.260 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom 2024-03-04T17:53:05.288 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom (6.8 kB at 234 kB/s) 2024-03-04T17:53:05.292 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom 2024-03-04T17:53:05.317 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom (8.4 kB at 323 kB/s) 2024-03-04T17:53:05.322 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom 2024-03-04T17:53:05.344 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom (2.1 kB at 98 kB/s) 2024-03-04T17:53:05.349 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom 2024-03-04T17:53:05.373 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom (2.2 kB at 93 kB/s) 2024-03-04T17:53:05.380 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom 2024-03-04T17:53:05.411 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom (3.0 kB at 98 kB/s) 2024-03-04T17:53:05.417 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom 2024-03-04T17:53:05.443 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom (15 kB at 576 kB/s) 2024-03-04T17:53:05.447 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom 2024-03-04T17:53:05.470 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom (15 kB at 631 kB/s) 2024-03-04T17:53:05.477 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom 2024-03-04T17:53:05.499 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom (13 kB at 611 kB/s) 2024-03-04T17:53:05.502 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom 2024-03-04T17:53:05.524 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom (4.8 kB at 228 kB/s) 2024-03-04T17:53:05.528 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom 2024-03-04T17:53:05.550 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom (6.3 kB at 299 kB/s) 2024-03-04T17:53:05.554 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom 2024-03-04T17:53:05.579 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/32 kB Progress (1): 32 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom (32 kB at 1.3 MB/s) 2024-03-04T17:53:05.585 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom 2024-03-04T17:53:05.608 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom (17 kB at 788 kB/s) 2024-03-04T17:53:05.612 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom 2024-03-04T17:53:05.634 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom (15 kB at 643 kB/s) 2024-03-04T17:53:05.640 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom 2024-03-04T17:53:05.663 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 750 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom (750 B at 34 kB/s) 2024-03-04T17:53:05.666 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom 2024-03-04T17:53:05.688 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom (4.8 kB at 218 kB/s) 2024-03-04T17:53:05.693 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom 2024-03-04T17:53:05.716 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 7.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom (7.9 kB at 344 kB/s) 2024-03-04T17:53:05.722 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom 2024-03-04T17:53:05.744 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom (3.0 kB at 136 kB/s) 2024-03-04T17:53:05.751 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom 2024-03-04T17:53:05.780 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom (4.2 kB at 147 kB/s) 2024-03-04T17:53:05.787 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom 2024-03-04T17:53:05.809 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom (2.1 kB at 98 kB/s) 2024-03-04T17:53:05.814 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom 2024-03-04T17:53:05.841 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom (3.5 kB at 134 kB/s) 2024-03-04T17:53:05.847 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom 2024-03-04T17:53:05.872 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom (11 kB at 421 kB/s) 2024-03-04T17:53:05.885 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom 2024-03-04T17:53:05.909 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom (13 kB at 561 kB/s) 2024-03-04T17:53:05.913 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom 2024-03-04T17:53:05.937 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/19 kB Progress (1): 19 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom (19 kB at 781 kB/s) 2024-03-04T17:53:05.944 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom 2024-03-04T17:53:05.967 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 363 B Downloaded from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom (363 B at 16 kB/s) 2024-03-04T17:53:05.971 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom 2024-03-04T17:53:05.994 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom (6.1 kB at 265 kB/s) 2024-03-04T17:53:05.997 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom 2024-03-04T17:53:06.020 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom (7.3 kB at 319 kB/s) 2024-03-04T17:53:06.025 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2024-03-04T17:53:06.049 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom (4.8 kB at 210 kB/s) 2024-03-04T17:53:06.056 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom 2024-03-04T17:53:06.079 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom (7.3 kB at 318 kB/s) 2024-03-04T17:53:06.085 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom 2024-03-04T17:53:06.110 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom (22 kB at 866 kB/s) 2024-03-04T17:53:06.115 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom 2024-03-04T17:53:06.148 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/78 kB Progress (1): 33/78 kB Progress (1): 49/78 kB Progress (1): 66/78 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom (78 kB at 2.4 MB/s) 2024-03-04T17:53:06.156 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom 2024-03-04T17:53:06.180 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom (23 kB at 1.0 MB/s) 2024-03-04T17:53:06.187 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom 2024-03-04T17:53:06.212 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom (5.6 kB at 217 kB/s) 2024-03-04T17:53:06.217 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom 2024-03-04T17:53:06.239 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom (4.3 kB at 194 kB/s) 2024-03-04T17:53:06.243 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom 2024-03-04T17:53:06.264 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom (3.2 kB at 145 kB/s) 2024-03-04T17:53:06.268 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom 2024-03-04T17:53:06.292 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/28 kB Progress (1): 28 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom (28 kB at 1.2 MB/s) 2024-03-04T17:53:06.298 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom 2024-03-04T17:53:06.320 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom (2.4 kB at 108 kB/s) 2024-03-04T17:53:06.332 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar 2024-03-04T17:53:06.389 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/308 kB Progress (1): 33/308 kB Progress (1): 49/308 kB Progress (1): 66/308 kB Progress (1): 82/308 kB Progress (1): 98/308 kB Progress (1): 115/308 kB Progress (1): 131/308 kB Progress (1): 147/308 kB Progress (1): 164/308 kB Progress (1): 180/308 kB Progress (1): 197/308 kB Progress (1): 213/308 kB Progress (1): 229/308 kB Progress (1): 246/308 kB Progress (1): 256/308 kB Progress (1): 272/308 kB Progress (1): 289/308 kB Progress (1): 305/308 kB Progress (1): 308 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar (308 kB at 5.4 MB/s) 2024-03-04T17:53:06.389 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar 2024-03-04T17:53:06.389 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar 2024-03-04T17:53:06.390 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar 2024-03-04T17:53:06.390 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar 2024-03-04T17:53:06.391 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar 2024-03-04T17:53:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/171 kB Progress (2): 16/171 kB | 8.2 kB Progress (2): 33/171 kB | 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar (8.2 kB at 327 kB/s) 2024-03-04T17:53:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar 2024-03-04T17:53:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 33/171 kB | 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar (14 kB at 469 kB/s) 2024-03-04T17:53:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar 2024-03-04T17:53:06.420 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 33/171 kB | 16/26 kB Progress (2): 49/171 kB | 16/26 kB Progress (2): 49/171 kB | 26 kB Progress (2): 66/171 kB | 26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar (26 kB at 862 kB/s) 2024-03-04T17:53:06.420 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar 2024-03-04T17:53:06.427 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 82/171 kB Progress (2): 82/171 kB | 16/118 kB Progress (2): 82/171 kB | 32/118 kB Progress (2): 82/171 kB | 49/118 kB Progress (2): 98/171 kB | 49/118 kB Progress (2): 98/171 kB | 65/118 kB Progress (2): 98/171 kB | 81/118 kB Progress (2): 98/171 kB | 98/118 kB Progress (2): 115/171 kB | 98/118 kB Progress (2): 115/171 kB | 114/118 kB Progress (2): 115/171 kB | 118 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar (118 kB at 3.2 MB/s) 2024-03-04T17:53:06.427 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 131/171 kB Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar 2024-03-04T17:53:06.432 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 147/171 kB Progress (1): 164/171 kB Progress (1): 171 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar (171 kB at 4.0 MB/s) 2024-03-04T17:53:06.432 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar 2024-03-04T17:53:06.444 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/146 kB Progress (2): 16/146 kB | 16/136 kB Progress (2): 32/146 kB | 16/136 kB Progress (3): 32/146 kB | 16/136 kB | 16/61 kB Progress (3): 32/146 kB | 16/136 kB | 33/61 kB Progress (3): 32/146 kB | 16/136 kB | 49/61 kB Progress (3): 32/146 kB | 16/136 kB | 61 kB Progress (3): 49/146 kB | 16/136 kB | 61 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar (61 kB at 1.1 MB/s) 2024-03-04T17:53:06.444 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar 2024-03-04T17:53:06.452 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 49/146 kB | 32/136 kB Progress (2): 65/146 kB | 32/136 kB Progress (2): 81/146 kB | 32/136 kB Progress (2): 81/146 kB | 49/136 kB Progress (2): 98/146 kB | 49/136 kB Progress (2): 114/146 kB | 49/136 kB Progress (2): 114/146 kB | 65/136 kB Progress (2): 114/146 kB | 81/136 kB Progress (2): 130/146 kB | 81/136 kB Progress (2): 146 kB | 81/136 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar (146 kB at 2.4 MB/s) 2024-03-04T17:53:06.453 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar 2024-03-04T17:53:06.460 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 81/136 kB | 16/501 kB Progress (2): 98/136 kB | 16/501 kB Progress (2): 98/136 kB | 33/501 kB Progress (2): 98/136 kB | 49/501 kB Progress (2): 98/136 kB | 66/501 kB Progress (2): 98/136 kB | 82/501 kB Progress (2): 114/136 kB | 82/501 kB Progress (2): 114/136 kB | 98/501 kB Progress (2): 114/136 kB | 115/501 kB Progress (2): 114/136 kB | 131/501 kB Progress (3): 114/136 kB | 131/501 kB | 16/58 kB Progress (3): 130/136 kB | 131/501 kB | 16/58 kB Progress (3): 130/136 kB | 147/501 kB | 16/58 kB Progress (3): 130/136 kB | 164/501 kB | 16/58 kB Progress (3): 130/136 kB | 180/501 kB | 16/58 kB Progress (3): 136 kB | 180/501 kB | 16/58 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar (136 kB at 1.9 MB/s) 2024-03-04T17:53:06.464 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 197/501 kB | 16/58 kB Progress (2): 197/501 kB | 33/58 kB Progress (2): 213/501 kB | 33/58 kB Progress (2): 229/501 kB | 33/58 kB Progress (2): 229/501 kB | 49/58 kB Progress (2): 246/501 kB | 49/58 kB Progress (2): 262/501 kB | 49/58 kB Progress (2): 262/501 kB | 58 kB Progress (2): 279/501 kB | 58 kB Progress (2): 295/501 kB | 58 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar (58 kB at 778 kB/s) 2024-03-04T17:53:06.472 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 311/501 kB Progress (1): 328/501 kB Progress (1): 344/501 kB Progress (1): 360/501 kB Progress (1): 377/501 kB Progress (1): 393/501 kB Progress (1): 410/501 kB Progress (1): 426/501 kB Progress (1): 442/501 kB Progress (1): 459/501 kB Progress (1): 475/501 kB Progress (1): 492/501 kB Progress (1): 501 kB Progress (2): 501 kB | 16/124 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar (501 kB at 6.2 MB/s) 2024-03-04T17:53:06.475 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 32/124 kB Progress (1): 49/124 kB Progress (1): 65/124 kB Progress (1): 81/124 kB Progress (1): 98/124 kB Progress (1): 114/124 kB Progress (1): 124 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar (124 kB at 1.5 MB/s) 2024-03-04T17:53:06.673 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 0/2.4 MB Progress (1): 0/2.4 MB Progress (1): 0/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar (2.4 MB at 8.6 MB/s) 2024-03-04T17:53:06.815 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:53:06.815 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] --- jar:3.3.0:jar (default-jar) @ jcksum --- 2024-03-04T17:53:06.819 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom 2024-03-04T17:53:06.843 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom (4.5 kB at 187 kB/s) 2024-03-04T17:53:06.846 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom 2024-03-04T17:53:06.873 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom (4.9 kB at 175 kB/s) 2024-03-04T17:53:06.875 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom 2024-03-04T17:53:06.906 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/45 kB Progress (1): 33/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom (45 kB at 1.5 MB/s) 2024-03-04T17:53:06.909 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom 2024-03-04T17:53:06.940 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom (21 kB at 663 kB/s) 2024-03-04T17:53:06.944 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom 2024-03-04T17:53:06.974 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 5.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom (5.2 kB at 167 kB/s) 2024-03-04T17:53:06.977 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom 2024-03-04T17:53:07.007 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 3.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom (3.9 kB at 131 kB/s) 2024-03-04T17:53:07.011 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom 2024-03-04T17:53:07.039 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom (6.0 kB at 207 kB/s) 2024-03-04T17:53:07.043 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom 2024-03-04T17:53:07.069 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom (6.3 kB at 232 kB/s) 2024-03-04T17:53:07.072 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom 2024-03-04T17:53:07.098 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom (20 kB at 757 kB/s) 2024-03-04T17:53:07.103 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom 2024-03-04T17:53:07.129 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom (15 kB at 581 kB/s) 2024-03-04T17:53:07.132 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom 2024-03-04T17:53:07.157 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom (2.0 kB at 82 kB/s) 2024-03-04T17:53:07.160 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom 2024-03-04T17:53:07.188 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom (8.2 kB at 293 kB/s) 2024-03-04T17:53:07.191 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom 2024-03-04T17:53:07.216 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom (25 kB at 1.0 MB/s) 2024-03-04T17:53:07.223 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar 2024-03-04T17:53:07.250 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/36 kB Progress (1): 33/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar (36 kB at 1.3 MB/s) 2024-03-04T17:53:07.251 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar 2024-03-04T17:53:07.251 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar 2024-03-04T17:53:07.251 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar 2024-03-04T17:53:07.252 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar 2024-03-04T17:53:07.252 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar 2024-03-04T17:53:07.281 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/79 kB Progress (1): 33/79 kB Progress (1): 49/79 kB Progress (1): 66/79 kB Progress (1): 79 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar (79 kB at 2.7 MB/s) 2024-03-04T17:53:07.281 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/26 kB Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar 2024-03-04T17:53:07.282 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar (26 kB at 875 kB/s) 2024-03-04T17:53:07.282 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar 2024-03-04T17:53:07.291 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/211 kB Progress (1): 33/211 kB Progress (2): 33/211 kB | 16/58 kB Progress (2): 49/211 kB | 16/58 kB Progress (2): 66/211 kB | 16/58 kB Progress (2): 66/211 kB | 33/58 kB Progress (2): 82/211 kB | 33/58 kB Progress (2): 98/211 kB | 33/58 kB Progress (3): 98/211 kB | 33/58 kB | 0/1.0 MB Progress (3): 115/211 kB | 33/58 kB | 0/1.0 MB Progress (3): 115/211 kB | 49/58 kB | 0/1.0 MB Progress (3): 115/211 kB | 58 kB | 0/1.0 MB Progress (3): 131/211 kB | 58 kB | 0/1.0 MB Progress (3): 147/211 kB | 58 kB | 0/1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar (58 kB at 1.4 MB/s) 2024-03-04T17:53:07.296 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 147/211 kB | 0/1.0 MB Progress (2): 164/211 kB | 0/1.0 MB Progress (2): 180/211 kB | 0/1.0 MB Progress (2): 180/211 kB | 0/1.0 MB Progress (2): 197/211 kB | 0/1.0 MB Progress (2): 197/211 kB | 0.1/1.0 MB Progress (2): 211 kB | 0.1/1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar (211 kB at 4.7 MB/s) 2024-03-04T17:53:07.316 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.2/1.0 MB Progress (2): 0.2/1.0 MB | 16/116 kB Progress (2): 0.2/1.0 MB | 32/116 kB Progress (2): 0.2/1.0 MB | 49/116 kB Progress (2): 0.2/1.0 MB | 65/116 kB Progress (2): 0.2/1.0 MB | 81/116 kB Progress (2): 0.2/1.0 MB | 98/116 kB Progress (3): 0.2/1.0 MB | 98/116 kB | 16/267 kB Progress (3): 0.2/1.0 MB | 98/116 kB | 32/267 kB Progress (3): 0.2/1.0 MB | 98/116 kB | 49/267 kB Progress (3): 0.2/1.0 MB | 114/116 kB | 49/267 kB Progress (3): 0.2/1.0 MB | 116 kB | 49/267 kB Progress (3): 0.2/1.0 MB | 116 kB | 65/267 kB Progress (3): 0.2/1.0 MB | 116 kB | 81/267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar (116 kB at 1.8 MB/s) 2024-03-04T17:53:07.320 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 0.2/1.0 MB | 98/267 kB Progress (2): 0.2/1.0 MB | 114/267 kB Progress (2): 0.2/1.0 MB | 130/267 kB Progress (2): 0.2/1.0 MB | 130/267 kB Progress (2): 0.2/1.0 MB | 147/267 kB Progress (2): 0.2/1.0 MB | 163/267 kB Progress (2): 0.2/1.0 MB | 180/267 kB Progress (2): 0.2/1.0 MB | 180/267 kB Progress (2): 0.2/1.0 MB | 196/267 kB Progress (2): 0.2/1.0 MB | 212/267 kB Progress (2): 0.2/1.0 MB | 229/267 kB Progress (2): 0.2/1.0 MB | 245/267 kB Progress (2): 0.2/1.0 MB | 262/267 kB Progress (2): 0.2/1.0 MB | 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar (267 kB at 3.9 MB/s) 2024-03-04T17:53:07.440 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 0.2/1.0 MB Progress (1): 0.2/1.0 MB Progress (1): 0.2/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar (1.0 MB at 5.4 MB/s) 2024-03-04T17:53:07.657 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Building jar: /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/target/jcksum-1.0.jar 2024-03-04T17:53:07.689 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] ------------------------------------------------------------------------ 2024-03-04T17:53:07.689 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] BUILD SUCCESS 2024-03-04T17:53:07.690 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] ------------------------------------------------------------------------ 2024-03-04T17:53:07.691 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Total time: 11.048 s 2024-03-04T17:53:07.691 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Finished at: 2024-03-04T17:53:07Z 2024-03-04T17:53:07.691 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] ------------------------------------------------------------------------ 2024-03-04T17:53:07.721 INFO:tasks.workunit.client.0.smithi137.stderr:+ ./mvnw test -Dtest=PutObjects 2024-03-04T17:53:07.727 INFO:tasks.workunit.client.0.smithi137.stdout:Warning: JAVA_HOME environment variable is not set. 2024-03-04T17:53:08.927 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Scanning for projects... 2024-03-04T17:53:09.048 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:53:09.048 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2024-03-04T17:53:09.048 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Building jcksum 1.0 2024-03-04T17:53:09.048 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] from pom.xml 2024-03-04T17:53:09.048 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2024-03-04T17:53:09.444 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:53:09.444 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2024-03-04T17:53:09.573 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2024-03-04T17:53:09.574 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:53:09.574 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2024-03-04T17:53:09.675 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Nothing to compile - all classes are up to date 2024-03-04T17:53:09.675 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:53:09.675 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2024-03-04T17:53:09.678 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2024-03-04T17:53:09.678 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:53:09.678 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2024-03-04T17:53:09.683 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Nothing to compile - all classes are up to date 2024-03-04T17:53:09.683 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:53:09.683 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2024-03-04T17:53:09.880 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider 2024-03-04T17:53:10.150 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom 2024-03-04T17:53:10.446 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.4/4.7 kB Progress (1): 2.8/4.7 kB Progress (1): 4.1/4.7 kB Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom (4.7 kB at 15 kB/s) 2024-03-04T17:53:10.456 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom 2024-03-04T17:53:10.497 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.4/2.6 kB Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom (2.6 kB at 62 kB/s) 2024-03-04T17:53:10.505 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom 2024-03-04T17:53:10.547 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.4/2.8 kB Progress (1): 2.8/2.8 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom (2.8 kB at 67 kB/s) 2024-03-04T17:53:10.555 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom 2024-03-04T17:53:10.597 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.4/3.0 kB Progress (1): 2.8/3.0 kB Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom (3.0 kB at 72 kB/s) 2024-03-04T17:53:10.601 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom 2024-03-04T17:53:10.643 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.4/3.2 kB Progress (1): 2.8/3.2 kB Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom (3.2 kB at 74 kB/s) 2024-03-04T17:53:10.647 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom 2024-03-04T17:53:10.690 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.4/1.7 kB Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom (1.7 kB at 40 kB/s) 2024-03-04T17:53:10.694 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom 2024-03-04T17:53:10.737 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.4/2.8 kB Progress (1): 2.8/2.8 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom (2.8 kB at 66 kB/s) 2024-03-04T17:53:10.748 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar 2024-03-04T17:53:10.806 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.4/27 kB Progress (1): 2.8/27 kB Progress (1): 4.1/27 kB Progress (1): 5.5/27 kB Progress (1): 6.9/27 kB Progress (1): 8.3/27 kB Progress (1): 9.7/27 kB Progress (1): 11/27 kB Progress (1): 12/27 kB Progress (1): 14/27 kB Progress (1): 15/27 kB Progress (1): 17/27 kB Progress (1): 18/27 kB Progress (1): 19/27 kB Progress (1): 21/27 kB Progress (1): 22/27 kB Progress (1): 23/27 kB Progress (1): 25/27 kB Progress (1): 26/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar (27 kB at 468 kB/s) 2024-03-04T17:53:10.808 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar 2024-03-04T17:53:10.809 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar 2024-03-04T17:53:10.810 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar 2024-03-04T17:53:10.810 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar 2024-03-04T17:53:10.811 INFO:tasks.workunit.client.0.smithi137.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar 2024-03-04T17:53:10.893 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/18 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar (18 kB at 202 kB/s) 2024-03-04T17:53:10.937 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 1.4/7.7 kB Progress (1): 2.8/7.7 kB Progress (1): 4.1/7.7 kB Progress (1): 5.5/7.7 kB Progress (1): 6.9/7.7 kB Progress (1): 7.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar (7.7 kB at 60 kB/s) 2024-03-04T17:53:11.028 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 16/189 kB Progress (2): 16/189 kB | 16/169 kB Progress (3): 16/189 kB | 16/169 kB | 1.4/103 kB Progress (3): 16/189 kB | 16/169 kB | 2.8/103 kB Progress (3): 33/189 kB | 16/169 kB | 2.8/103 kB Progress (3): 33/189 kB | 16/169 kB | 4.1/103 kB Progress (3): 33/189 kB | 33/169 kB | 4.1/103 kB Progress (3): 33/189 kB | 33/169 kB | 5.5/103 kB Progress (3): 33/189 kB | 33/169 kB | 6.9/103 kB Progress (3): 33/189 kB | 33/169 kB | 8.3/103 kB Progress (3): 33/189 kB | 33/169 kB | 9.7/103 kB Progress (3): 33/189 kB | 33/169 kB | 11/103 kB Progress (3): 33/189 kB | 33/169 kB | 12/103 kB Progress (3): 33/189 kB | 33/169 kB | 14/103 kB Progress (3): 33/189 kB | 33/169 kB | 15/103 kB Progress (3): 33/189 kB | 33/169 kB | 17/103 kB Progress (3): 33/189 kB | 33/169 kB | 18/103 kB Progress (3): 33/189 kB | 33/169 kB | 19/103 kB Progress (3): 33/189 kB | 33/169 kB | 21/103 kB Progress (3): 33/189 kB | 33/169 kB | 22/103 kB Progress (3): 33/189 kB | 33/169 kB | 23/103 kB Progress (3): 33/189 kB | 33/169 kB | 25/103 kB Progress (3): 33/189 kB | 33/169 kB | 26/103 kB Progress (3): 33/189 kB | 33/169 kB | 28/103 kB Progress (3): 33/189 kB | 33/169 kB | 29/103 kB Progress (3): 33/189 kB | 33/169 kB | 30/103 kB Progress (3): 33/189 kB | 33/169 kB | 32/103 kB Progress (3): 33/189 kB | 33/169 kB | 33/103 kB Progress (3): 33/189 kB | 33/169 kB | 34/103 kB Progress (3): 33/189 kB | 33/169 kB | 36/103 kB Progress (3): 33/189 kB | 33/169 kB | 37/103 kB Progress (3): 33/189 kB | 49/169 kB | 37/103 kB Progress (3): 49/189 kB | 49/169 kB | 37/103 kB Progress (3): 49/189 kB | 66/169 kB | 37/103 kB Progress (3): 66/189 kB | 66/169 kB | 37/103 kB Progress (3): 82/189 kB | 66/169 kB | 37/103 kB Progress (3): 82/189 kB | 82/169 kB | 37/103 kB Progress (3): 82/189 kB | 82/169 kB | 53/103 kB Progress (3): 82/189 kB | 82/169 kB | 69/103 kB Progress (3): 82/189 kB | 98/169 kB | 69/103 kB Progress (3): 98/189 kB | 98/169 kB | 69/103 kB Progress (3): 98/189 kB | 115/169 kB | 69/103 kB Progress (3): 115/189 kB | 115/169 kB | 69/103 kB Progress (3): 115/189 kB | 131/169 kB | 69/103 kB Progress (3): 131/189 kB | 131/169 kB | 69/103 kB Progress (3): 131/189 kB | 147/169 kB | 69/103 kB Progress (3): 147/189 kB | 147/169 kB | 69/103 kB Progress (3): 147/189 kB | 164/169 kB | 69/103 kB Progress (3): 164/189 kB | 164/169 kB | 69/103 kB Progress (3): 164/189 kB | 169 kB | 69/103 kB Progress (3): 164/189 kB | 169 kB | 86/103 kB Progress (3): 180/189 kB | 169 kB | 86/103 kB Progress (3): 189 kB | 169 kB | 86/103 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar (169 kB at 769 kB/s) 2024-03-04T17:53:11.030 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (2): 189 kB | 102/103 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar (189 kB at 856 kB/s) 2024-03-04T17:53:11.032 INFO:tasks.workunit.client.0.smithi137.stdout:Progress (1): 103 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar (103 kB at 462 kB/s) 2024-03-04T17:53:11.067 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:53:11.067 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] ------------------------------------------------------- 2024-03-04T17:53:11.067 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] T E S T S 2024-03-04T17:53:11.067 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] ------------------------------------------------------- 2024-03-04T17:53:11.684 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Running io.ceph.jcksum.PutObjects 2024-03-04T17:53:11.778 INFO:tasks.workunit.client.0.smithi137.stdout:PutObjects.java: starting test run: 2024-03-04T17:53:11.778 INFO:tasks.workunit.client.0.smithi137.stdout: AccessKey=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-03-04T17:53:11.778 INFO:tasks.workunit.client.0.smithi137.stdout: SecretKey=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-03-04T17:53:11.779 INFO:tasks.workunit.client.0.smithi137.stdout: EndpointUrl=https://smithi137.front.sepia.ceph.com:443 2024-03-04T17:53:11.779 INFO:tasks.workunit.client.0.smithi137.stderr:[main] WARN software.amazon.awssdk.http.apache.ApacheHttpClient - SSL Certificate verification is disabled. This is not a safe setting and should only be used for testing. 2024-03-04T17:53:11.873 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: Environment Variables Read 2024-03-04T17:53:12.275 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: S3 Client Initialized 2024-03-04T17:53:12.275 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: Generating File 2024-03-04T17:53:12.275 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: File Generated 2024-03-04T17:53:12.275 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: Generating File 2024-03-04T17:53:12.275 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: File Generated 2024-03-04T17:53:12.275 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: Generating File 2024-03-04T17:53:12.374 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: File Generated 2024-03-04T17:53:12.375 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: Generating File 2024-03-04T17:53:12.375 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: File Generated 2024-03-04T17:53:12.376 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: Generating File 2024-03-04T17:53:12.376 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: File Generated 2024-03-04T17:53:12.376 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: Generating File 2024-03-04T17:53:12.376 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: File Generated 2024-03-04T17:53:12.376 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: Generated Big Files 2024-03-04T17:53:12.949 INFO:tasks.workunit.client.0.smithi137.stdout:HeadBucketResponse() 2024-03-04T17:53:12.949 INFO:tasks.workunit.client.0.smithi137.stdout:sheik is ready 2024-03-04T17:53:12.949 INFO:tasks.workunit.client.0.smithi137.stdout:DEBUG: Test Bucket Created 2024-03-04T17:53:13.076 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileNoCksum called with file-0b 2024-03-04T17:53:13.377 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileNoCksum called with file-8b 2024-03-04T17:53:13.577 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileNoCksum called with file-200b 2024-03-04T17:53:13.978 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileNoCksum called with file-21983b 2024-03-04T17:53:14.179 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileNoCksum called with file-5519b 2024-03-04T17:53:14.480 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileNoCksum called with file-204329b 2024-03-04T17:53:15.081 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileNoCksum called with file-256k 2024-03-04T17:53:15.582 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileNoCksum called with file-1m 2024-03-04T17:53:16.584 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileNoCksum called with file-1038757b 2024-03-04T17:53:17.586 INFO:tasks.workunit.client.0.smithi137.stdout:mpuObjectFromFileNoCksum called with file-5m 2024-03-04T17:53:23.800 INFO:tasks.workunit.client.0.smithi137.stdout:mpuObjectFromFileNoCksum called with file-10m 2024-03-04T17:53:31.414 INFO:tasks.workunit.client.0.smithi137.stdout:mpuObjectFromFileNoCksum called with file-100m 2024-03-04T17:54:38.247 INFO:tasks.workunit.client.0.smithi137.stdout:arg: file-0b 2024-03-04T17:54:38.248 INFO:tasks.workunit.client.0.smithi137.stdout:arg: file-8b 2024-03-04T17:54:38.249 INFO:tasks.workunit.client.0.smithi137.stdout:arg: file-200b 2024-03-04T17:54:38.250 INFO:tasks.workunit.client.0.smithi137.stdout:arg: file-21983b 2024-03-04T17:54:38.251 INFO:tasks.workunit.client.0.smithi137.stdout:arg: file-5519b 2024-03-04T17:54:38.252 INFO:tasks.workunit.client.0.smithi137.stdout:arg: file-204329b 2024-03-04T17:54:38.253 INFO:tasks.workunit.client.0.smithi137.stdout:arg: file-256k 2024-03-04T17:54:38.254 INFO:tasks.workunit.client.0.smithi137.stdout:arg: file-1m 2024-03-04T17:54:38.255 INFO:tasks.workunit.client.0.smithi137.stdout:arg: file-1038757b 2024-03-04T17:54:38.309 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileCksum called with file-0b 2024-03-04T17:54:38.509 INFO:tasks.workunit.client.0.smithi137.stdout:PutObjectResponse 2024-03-04T17:54:38.609 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileCksum called with file-8b 2024-03-04T17:54:38.709 INFO:tasks.workunit.client.0.smithi137.stdout:PutObjectResponse 2024-03-04T17:54:38.810 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileCksum called with file-200b 2024-03-04T17:54:38.910 INFO:tasks.workunit.client.0.smithi137.stdout:PutObjectResponse 2024-03-04T17:54:39.110 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileCksum called with file-21983b 2024-03-04T17:54:39.210 INFO:tasks.workunit.client.0.smithi137.stdout:PutObjectResponse 2024-03-04T17:54:39.310 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileCksum called with file-5519b 2024-03-04T17:54:39.410 INFO:tasks.workunit.client.0.smithi137.stdout:PutObjectResponse 2024-03-04T17:54:39.511 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileCksum called with file-204329b 2024-03-04T17:54:39.811 INFO:tasks.workunit.client.0.smithi137.stdout:PutObjectResponse 2024-03-04T17:54:40.012 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileCksum called with file-256k 2024-03-04T17:54:40.312 INFO:tasks.workunit.client.0.smithi137.stdout:PutObjectResponse 2024-03-04T17:54:40.412 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileCksum called with file-1m 2024-03-04T17:54:41.213 INFO:tasks.workunit.client.0.smithi137.stdout:PutObjectResponse 2024-03-04T17:54:41.514 INFO:tasks.workunit.client.0.smithi137.stdout:putObjectFromFileCksum called with file-1038757b 2024-03-04T17:54:42.314 INFO:tasks.workunit.client.0.smithi137.stdout:PutObjectResponse 2024-03-04T17:54:42.615 INFO:tasks.workunit.client.0.smithi137.stdout:mpuObjectFromFileCksum called with file-5m 2024-03-04T17:54:47.521 INFO:tasks.workunit.client.0.smithi137.stdout:mpuObjectFromFileCksum called with file-10m 2024-03-04T17:54:58.634 INFO:tasks.workunit.client.0.smithi137.stdout:mpuObjectFromFileCksum called with file-100m 2024-03-04T17:56:46.400 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 214.7 s -- in io.ceph.jcksum.PutObjects 2024-03-04T17:56:46.424 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:56:46.424 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Results: 2024-03-04T17:56:46.424 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:56:46.424 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0 2024-03-04T17:56:46.425 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] 2024-03-04T17:56:46.427 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] ------------------------------------------------------------------------ 2024-03-04T17:56:46.427 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] BUILD SUCCESS 2024-03-04T17:56:46.427 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] ------------------------------------------------------------------------ 2024-03-04T17:56:46.428 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Total time: 03:37 min 2024-03-04T17:56:46.428 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] Finished at: 2024-03-04T17:56:46Z 2024-03-04T17:56:46.428 INFO:tasks.workunit.client.0.smithi137.stdout:[INFO] ------------------------------------------------------------------------ 2024-03-04T17:56:46.450 INFO:tasks.workunit.client.0.smithi137.stderr:+ exit 2024-03-04T17:56:46.450 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:56:46.451 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:56:46.504 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh'] on client.0... 2024-03-04T17:56:46.504 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-04T17:56:46.869 DEBUG:teuthology.parallel:result is None 2024-03-04T17:56:46.869 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:56:46.881 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:56:46.881 DEBUG:teuthology.orchestra.run.smithi137:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:56:46.929 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:56:46.929 INFO:teuthology.run_tasks:Running task workunit... 2024-03-04T17:56:46.939 INFO:tasks.workunit:Pulling workunits from ref 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:56:46.939 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-04T17:56:46.939 DEBUG:teuthology.orchestra.run.smithi137:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:56:46.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T17:56:46.977 INFO:teuthology.orchestra.run.smithi137.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-04T17:56:46.978 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:56:47.025 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-04T17:56:47.025 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-04T17:56:47.073 INFO:tasks.workunit:timeout=3h 2024-03-04T17:56:47.073 INFO:tasks.workunit:cleanup=True 2024-03-04T17:56:47.074 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/pritha-srivastava/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T17:56:47.123 INFO:tasks.workunit.client.0.smithi137.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-04T17:57:40.981 INFO:tasks.workunit.client.0.smithi137.stderr:Note: switching to '3aec390336fdfdc8f927818b718cfb0361bd47c8'. 2024-03-04T17:57:40.982 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:57:40.982 INFO:tasks.workunit.client.0.smithi137.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T17:57:40.982 INFO:tasks.workunit.client.0.smithi137.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T17:57:40.982 INFO:tasks.workunit.client.0.smithi137.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T17:57:40.982 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:57:40.982 INFO:tasks.workunit.client.0.smithi137.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T17:57:40.982 INFO:tasks.workunit.client.0.smithi137.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T17:57:40.982 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:57:40.982 INFO:tasks.workunit.client.0.smithi137.stderr: git switch -c 2024-03-04T17:57:40.982 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:57:40.983 INFO:tasks.workunit.client.0.smithi137.stderr:Or undo this operation with: 2024-03-04T17:57:40.983 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:57:40.983 INFO:tasks.workunit.client.0.smithi137.stderr: git switch - 2024-03-04T17:57:40.983 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:57:40.983 INFO:tasks.workunit.client.0.smithi137.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T17:57:40.983 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T17:57:40.983 INFO:tasks.workunit.client.0.smithi137.stderr:HEAD is now at 3aec390336f qa/d4n: Start redis up before rgw 2024-03-04T17:57:40.989 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-04T17:57:41.034 INFO:tasks.workunit.client.0.smithi137.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T17:57:41.038 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T17:57:41.038 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T17:57:41.105 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T17:57:41.164 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T17:57:41.209 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T17:57:41.210 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T17:57:41.211 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T17:57:41.256 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T17:57:41.260 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:57:41.260 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-04T17:57:41.305 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_s3_mp_reupload.sh on client.0... 2024-03-04T17:57:41.306 INFO:tasks.workunit:Running workunit rgw/test_rgw_s3_mp_reupload.sh... 2024-03-04T17:57:41.307 DEBUG:teuthology.orchestra.run.smithi137:workunit test rgw/test_rgw_s3_mp_reupload.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh 2024-03-04T17:57:41.360 INFO:tasks.workunit.client.0.smithi137.stdout:INFO: fully qualified domain name: smithi137.front.sepia.ceph.com 2024-03-04T17:57:41.360 INFO:tasks.workunit.client.0.smithi137.stdout:== TESTING *COMPLETING* MULTIPART UPLOAD WITH RE-UPLOADS == 2024-03-04T17:57:41.360 INFO:tasks.workunit.client.0.smithi137.stdout:== TESTING VERSIONED BUCKET == 2024-03-04T17:57:44.574 INFO:tasks.workunit.client.0.smithi137.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-03-04T17:57:44.920 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting pip 2024-03-04T17:57:44.927 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-03-04T17:57:44.962 INFO:tasks.workunit.client.0.smithi137.stdout:Installing collected packages: pip 2024-03-04T17:57:44.963 INFO:tasks.workunit.client.0.smithi137.stdout: Attempting uninstall: pip 2024-03-04T17:57:44.963 INFO:tasks.workunit.client.0.smithi137.stdout: Found existing installation: pip 22.0.2 2024-03-04T17:57:45.222 INFO:tasks.workunit.client.0.smithi137.stdout: Uninstalling pip-22.0.2: 2024-03-04T17:57:45.228 INFO:tasks.workunit.client.0.smithi137.stdout: Successfully uninstalled pip-22.0.2 2024-03-04T17:57:46.267 INFO:tasks.workunit.client.0.smithi137.stdout:Successfully installed pip-24.0 2024-03-04T17:57:47.395 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting boto3 2024-03-04T17:57:47.396 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached boto3-1.34.54-py3-none-any.whl.metadata (6.6 kB) 2024-03-04T17:57:47.931 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting botocore<1.35.0,>=1.34.54 (from boto3) 2024-03-04T17:57:47.932 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached botocore-1.34.54-py3-none-any.whl.metadata (5.7 kB) 2024-03-04T17:57:48.016 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-03-04T17:57:48.017 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-03-04T17:57:48.112 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-03-04T17:57:48.114 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached s3transfer-0.10.0-py3-none-any.whl.metadata (1.7 kB) 2024-03-04T17:57:48.240 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T17:57:48.243 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-03-04T17:57:48.411 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting urllib3<2.1,>=1.25.4 (from botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T17:57:48.413 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB) 2024-03-04T17:57:48.594 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T17:57:48.595 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-03-04T17:57:48.609 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached boto3-1.34.54-py3-none-any.whl (139 kB) 2024-03-04T17:57:48.611 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached botocore-1.34.54-py3-none-any.whl (12.0 MB) 2024-03-04T17:57:48.633 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-03-04T17:57:48.634 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached s3transfer-0.10.0-py3-none-any.whl (82 kB) 2024-03-04T17:57:48.636 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-03-04T17:57:48.637 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached urllib3-2.0.7-py3-none-any.whl (124 kB) 2024-03-04T17:57:48.639 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-03-04T17:57:48.739 INFO:tasks.workunit.client.0.smithi137.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-03-04T17:57:49.897 INFO:tasks.workunit.client.0.smithi137.stdout:Successfully installed boto3-1.34.54 botocore-1.34.54 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.0 six-1.16.0 urllib3-2.0.7 2024-03-04T17:57:50.450 INFO:tasks.workunit.client.0.smithi137.stdout:INFO: creating user 2024-03-04T17:57:50.821 INFO:tasks.workunit.client.0.smithi137.stdout:{ 2024-03-04T17:57:50.821 INFO:tasks.workunit.client.0.smithi137.stdout: "user_id": "testid", 2024-03-04T17:57:50.821 INFO:tasks.workunit.client.0.smithi137.stdout: "display_name": "M. Tester", 2024-03-04T17:57:50.821 INFO:tasks.workunit.client.0.smithi137.stdout: "email": "tester@ceph.com", 2024-03-04T17:57:50.821 INFO:tasks.workunit.client.0.smithi137.stdout: "suspended": 0, 2024-03-04T17:57:50.821 INFO:tasks.workunit.client.0.smithi137.stdout: "max_buckets": 1000, 2024-03-04T17:57:50.822 INFO:tasks.workunit.client.0.smithi137.stdout: "subusers": [], 2024-03-04T17:57:50.822 INFO:tasks.workunit.client.0.smithi137.stdout: "keys": [ 2024-03-04T17:57:50.822 INFO:tasks.workunit.client.0.smithi137.stdout: { 2024-03-04T17:57:50.822 INFO:tasks.workunit.client.0.smithi137.stdout: "user": "testid", 2024-03-04T17:57:50.822 INFO:tasks.workunit.client.0.smithi137.stdout: "access_key": "0555b35654ad1656d804", 2024-03-04T17:57:50.822 INFO:tasks.workunit.client.0.smithi137.stdout: "secret_key": "h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==", 2024-03-04T17:57:50.822 INFO:tasks.workunit.client.0.smithi137.stdout: "active": true 2024-03-04T17:57:50.822 INFO:tasks.workunit.client.0.smithi137.stdout: } 2024-03-04T17:57:50.822 INFO:tasks.workunit.client.0.smithi137.stdout: ], 2024-03-04T17:57:50.822 INFO:tasks.workunit.client.0.smithi137.stdout: "swift_keys": [], 2024-03-04T17:57:50.822 INFO:tasks.workunit.client.0.smithi137.stdout: "caps": [], 2024-03-04T17:57:50.822 INFO:tasks.workunit.client.0.smithi137.stdout: "op_mask": "read, write, delete", 2024-03-04T17:57:50.822 INFO:tasks.workunit.client.0.smithi137.stdout: "default_placement": "", 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "default_storage_class": "", 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "placement_tags": [], 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "bucket_quota": { 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "enabled": false, 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "check_on_raw": false, 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "max_size": -1, 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "max_size_kb": 0, 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "max_objects": -1 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "user_quota": { 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "enabled": false, 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "check_on_raw": false, 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "max_size": -1, 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "max_size_kb": 0, 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "max_objects": -1 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: }, 2024-03-04T17:57:50.823 INFO:tasks.workunit.client.0.smithi137.stdout: "temp_url_keys": [], 2024-03-04T17:57:50.824 INFO:tasks.workunit.client.0.smithi137.stdout: "type": "rgw", 2024-03-04T17:57:50.824 INFO:tasks.workunit.client.0.smithi137.stdout: "mfa_ids": [] 2024-03-04T17:57:50.824 INFO:tasks.workunit.client.0.smithi137.stdout:} 2024-03-04T17:57:50.824 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-03-04T17:58:02.283 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:02.284 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:02.400 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:02.400 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:02.488 INFO:tasks.workunit.client.0.smithi137.stderr:5+0 records in 2024-03-04T17:58:02.488 INFO:tasks.workunit.client.0.smithi137.stderr:5+0 records out 2024-03-04T17:58:02.488 INFO:tasks.workunit.client.0.smithi137.stderr:5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.0217812 s, 241 MB/s 2024-03-04T17:58:02.493 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:02.494 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:02.612 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:02.612 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:05.015 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:05.015 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:07.407 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:07.407 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:09.832 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:09.832 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:12.236 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:12.236 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:14.577 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:14.577 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:16.979 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:16.980 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:19.377 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:19.377 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:21.765 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:21.765 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:24.151 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:24.152 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:26.569 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:26.570 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:28.948 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:28.948 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:31.349 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:31.349 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:33.707 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi137.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T17:58:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T17:58:34.001 INFO:tasks.workunit.client.0.smithi137.stdout:bucket nams is reupload-bkt-108675 2024-03-04T17:58:34.001 INFO:tasks.workunit.client.0.smithi137.stdout:endpoint is https://smithi137.front.sepia.ceph.com:443 2024-03-04T17:58:34.001 INFO:tasks.workunit.client.0.smithi137.stdout:start UploadId=2~LOL7ea4Rq7amp5gEgA06uXztAuBlr7w 2024-03-04T17:58:34.001 INFO:tasks.workunit.client.0.smithi137.stdout:phase 1 uploaded part {'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 1} 2024-03-04T17:58:34.001 INFO:tasks.workunit.client.0.smithi137.stdout:phase 1 uploaded part {'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 2} 2024-03-04T17:58:34.002 INFO:tasks.workunit.client.0.smithi137.stdout:phase 2 uploaded part {'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 1} 2024-03-04T17:58:34.002 INFO:tasks.workunit.client.0.smithi137.stdout:phase 2 uploaded part {'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 1} 2024-03-04T17:58:34.002 INFO:tasks.workunit.client.0.smithi137.stdout:phase 2 uploaded part {'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 1} 2024-03-04T17:58:34.002 INFO:tasks.workunit.client.0.smithi137.stdout:phase 2 uploaded part {'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 1} 2024-03-04T17:58:34.002 INFO:tasks.workunit.client.0.smithi137.stdout:phase 2 uploaded part {'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 1} 2024-03-04T17:58:34.002 INFO:tasks.workunit.client.0.smithi137.stdout:phase 2 uploaded part {'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 1} 2024-03-04T17:58:34.002 INFO:tasks.workunit.client.0.smithi137.stdout:phase 2 uploaded part {'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 1} 2024-03-04T17:58:34.002 INFO:tasks.workunit.client.0.smithi137.stdout:phase 2 uploaded part {'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 1} 2024-03-04T17:58:34.002 INFO:tasks.workunit.client.0.smithi137.stdout:phase 2 uploaded part {'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 1} 2024-03-04T17:58:34.002 INFO:tasks.workunit.client.0.smithi137.stdout:phase 2 uploaded part {'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 1} 2024-03-04T17:58:34.002 INFO:tasks.workunit.client.0.smithi137.stdout:phase 2 uploaded part {'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 1} 2024-03-04T17:58:34.002 INFO:tasks.workunit.client.0.smithi137.stdout:completing multipart upload, parts=[{'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 1}, {'ETag': '"ee1d0e4db2ed63b453aec0d8c70ea930"', 'PartNumber': 2}] 2024-03-04T17:58:35.699 INFO:tasks.workunit.client.0.smithi137.stdout:radosgw-admin bucket radoslist: 2024-03-04T17:58:35.700 INFO:tasks.workunit.client.0.smithi137.stdout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4__:21kxpM7LJeN6.ujUpnMPXp4M.WEMDMh_mpu_test4 2024-03-04T17:58:35.700 INFO:tasks.workunit.client.0.smithi137.stdout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4_mpu_test4 2024-03-04T17:58:35.700 INFO:tasks.workunit.client.0.smithi137.stdout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4__multipart_mpu_test4.2~LOL7ea4Rq7amp5gEgA06uXztAuBlr7w.2 2024-03-04T17:58:35.700 INFO:tasks.workunit.client.0.smithi137.stdout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4__multipart_mpu_test4.OEpt0OuyPI73E2KAjK-dIqihs-4153uU.1 2024-03-04T17:58:35.700 INFO:tasks.workunit.client.0.smithi137.stdout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4__shadow_mpu_test4.2~LOL7ea4Rq7amp5gEgA06uXztAuBlr7w.2_1 2024-03-04T17:58:35.700 INFO:tasks.workunit.client.0.smithi137.stdout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4__shadow_mpu_test4.OEpt0OuyPI73E2KAjK-dIqihs-4153uU.1_1 2024-03-04T17:58:35.782 INFO:tasks.workunit.client.0.smithi137.stdout:rados ls: 2024-03-04T17:58:35.783 INFO:tasks.workunit.client.0.smithi137.stdout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4__:21kxpM7LJeN6.ujUpnMPXp4M.WEMDMh_mpu_test4 2024-03-04T17:58:35.783 INFO:tasks.workunit.client.0.smithi137.stdout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4_mpu_test4 2024-03-04T17:58:35.783 INFO:tasks.workunit.client.0.smithi137.stdout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4__multipart_mpu_test4.2~LOL7ea4Rq7amp5gEgA06uXztAuBlr7w.2 2024-03-04T17:58:35.783 INFO:tasks.workunit.client.0.smithi137.stdout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4__multipart_mpu_test4.OEpt0OuyPI73E2KAjK-dIqihs-4153uU.1 2024-03-04T17:58:35.783 INFO:tasks.workunit.client.0.smithi137.stdout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4__shadow_mpu_test4.2~LOL7ea4Rq7amp5gEgA06uXztAuBlr7w.2_1 2024-03-04T17:58:35.783 INFO:tasks.workunit.client.0.smithi137.stdout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4__shadow_mpu_test4.OEpt0OuyPI73E2KAjK-dIqihs-4153uU.1_1 2024-03-04T17:58:35.802 INFO:tasks.workunit.client.0.smithi137.stdout:== PASSED == 2024-03-04T17:58:35.802 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T17:58:35.803 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T17:58:35.860 INFO:tasks.workunit:Stopping ['rgw/test_rgw_s3_mp_reupload.sh'] on client.0... 2024-03-04T17:58:35.860 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-04T17:58:36.287 DEBUG:teuthology.parallel:result is None 2024-03-04T17:58:36.287 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:58:36.299 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:58:36.300 DEBUG:teuthology.orchestra.run.smithi137:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T17:58:36.346 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T17:58:36.346 INFO:teuthology.run_tasks:Running task ragweed... 2024-03-04T17:58:36.363 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'default-branch': 'ceph-master', 'rgw_server': 'client.0', 'stages': 'prepare,check'}} 2024-03-04T17:58:36.363 INFO:tasks.ragweed:Downloading ragweed... 2024-03-04T17:58:36.363 INFO:tasks.ragweed:Using branch 'wip-rgw-d4n' for ragweed 2024-03-04T17:58:36.363 DEBUG:teuthology.orchestra.run.smithi137:> git clone -b wip-rgw-d4n https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2024-03-04T17:58:36.394 INFO:teuthology.orchestra.run.smithi137.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2024-03-04T17:58:36.448 INFO:teuthology.orchestra.run.smithi137.stderr:fatal: Remote branch wip-rgw-d4n not found in upstream origin 2024-03-04T17:58:36.449 DEBUG:teuthology.orchestra.run:got remote process result: 128 2024-03-04T17:58:36.450 INFO:tasks.ragweed:Using branch 'ceph-master' for ragweed 2024-03-04T17:58:36.450 DEBUG:teuthology.orchestra.run.smithi137:> git clone -b ceph-master https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2024-03-04T17:58:36.499 INFO:teuthology.orchestra.run.smithi137.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2024-03-04T17:58:36.675 INFO:tasks.ragweed:Creating rgw users... 2024-03-04T17:58:36.675 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2024-03-04T17:58:36.675 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key QOXRJIHOSDFWAPOBUJVJ --secret EGiv4KsMCxchNkaPE1FJgMpIr9berR4jiv8GJhkzQxcih+cr7hjSFA== --email ragweed.client.0+test@test.test --system false 2024-03-04T17:58:36.764 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:58:36.764 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:58:36.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:36.805+0000 7f97cd1ec980 20 rados->read ofs=0 len=0 2024-03-04T17:58:36.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:36.805+0000 7f97cd1ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:36.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:36.805+0000 7f97cd1ec980 20 realm 2024-03-04T17:58:36.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:36.805+0000 7f97cd1ec980 20 rados->read ofs=0 len=0 2024-03-04T17:58:36.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:36.809+0000 7f97cd1ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:36.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:36.809+0000 7f97cd1ec980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:58:36.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:36.809+0000 7f97cd1ec980 20 rados->read ofs=0 len=0 2024-03-04T17:58:36.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:36.809+0000 7f97cd1ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:36.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:36.809+0000 7f97cd1ec980 20 rados->read ofs=0 len=0 2024-03-04T17:58:36.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:36.809+0000 7f97cd1ec980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:58:36.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:36.809+0000 7f97cd1ec980 20 rados->read ofs=0 len=0 2024-03-04T17:58:36.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:36.809+0000 7f97cd1ec980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:58:36.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:36.809+0000 7f97cd1ec980 20 searching for the correct realm 2024-03-04T17:58:36.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:36.813+0000 7f97cb505640 0 -- 172.21.15.137:0/1115628874 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f97c40b63f0 msgr2=0x7f97c40b6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:58:37.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.029+0000 7f97cb505640 0 -- 172.21.15.137:0/1115628874 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55affbaef3f0 msgr2=0x55affbaf18b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:58:37.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.037+0000 7f97cd1ec980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:58:37.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.037+0000 7f97cd1ec980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:58:37.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.037+0000 7f97cd1ec980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:58:37.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.037+0000 7f97cd1ec980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:58:37.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.037+0000 7f97cd1ec980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:58:37.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.037+0000 7f97cd1ec980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:58:37.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.037+0000 7f97cd1ec980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.037+0000 7f97cd1ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.037+0000 7f97cd1ec980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.037+0000 7f97cd1ec980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:58:37.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.037+0000 7f97cd1ec980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:58:37.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 20 zone default found 2024-03-04T17:58:37.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 4 Realm: () 2024-03-04T17:58:37.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:58:37.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:58:37.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:58:37.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 20 zonegroup default 2024-03-04T17:58:37.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 20 started sync module instance, tier type = 2024-03-04T17:58:37.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.041+0000 7f97cd1ec980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:58:37.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.065+0000 7f97cd1ec980 20 add_watcher() i=0 2024-03-04T17:58:37.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.065+0000 7f97cd1ec980 20 add_watcher() i=1 2024-03-04T17:58:37.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.065+0000 7f97cd1ec980 20 add_watcher() i=2 2024-03-04T17:58:37.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.065+0000 7f97cd1ec980 20 add_watcher() i=3 2024-03-04T17:58:37.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.065+0000 7f97cd1ec980 20 add_watcher() i=4 2024-03-04T17:58:37.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.065+0000 7f97cd1ec980 20 add_watcher() i=5 2024-03-04T17:58:37.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.065+0000 7f97cd1ec980 20 add_watcher() i=6 2024-03-04T17:58:37.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.065+0000 7f97cd1ec980 20 add_watcher() i=7 2024-03-04T17:58:37.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.065+0000 7f97cd1ec980 2 all 8 watchers are set, enabling cache 2024-03-04T17:58:37.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.069+0000 7f96817e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:58:37.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.069+0000 7f97cd1ec980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:58:37.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.069+0000 7f97cd1ec980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:58:37.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.069+0000 7f97cd1ec980 5 note: GC not initialized 2024-03-04T17:58:37.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f95fbfff640 20 reqs_thread_entry: start 2024-03-04T17:58:37.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 20 init_complete bucket index max shards: 1 2024-03-04T17:58:37.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 20 Filter name: none 2024-03-04T17:58:37.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f95f9ffb640 20 reqs_thread_entry: start 2024-03-04T17:58:37.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2024-03-04T17:58:37.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2024-03-04T17:58:37.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-03-04T17:58:37.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2024-03-04T17:58:37.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2024-03-04T17:58:37.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-03-04T17:58:37.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 10 cache get: name=default.rgw.meta+users.keys+QOXRJIHOSDFWAPOBUJVJ : miss 2024-03-04T17:58:37.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 10 cache put: name=default.rgw.meta+users.keys+QOXRJIHOSDFWAPOBUJVJ info.flags=0x0 2024-03-04T17:58:37.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 10 adding default.rgw.meta+users.keys+QOXRJIHOSDFWAPOBUJVJ to cache LRU end 2024-03-04T17:58:37.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 10 cache get: name=default.rgw.meta+users.keys+QOXRJIHOSDFWAPOBUJVJ : hit (negative entry) 2024-03-04T17:58:37.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.073+0000 7f97cd1ec980 10 cache get: name=default.rgw.meta+users.keys+QOXRJIHOSDFWAPOBUJVJ : hit (negative entry) 2024-03-04T17:58:37.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.077+0000 7f97cd1ec980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2024-03-04T17:58:37.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.077+0000 7f97cd1ec980 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-03-04T17:58:37.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.077+0000 7f97cd1ec980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2024-03-04T17:58:37.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.077+0000 7f96ae7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824068 cookie 94214330207904 notifier 5573 bl.length()=642 2024-03-04T17:58:37.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.077+0000 7f96ae7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2024-03-04T17:58:37.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.077+0000 7f96ae7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-03-04T17:58:37.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.081+0000 7f97cd1ec980 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2024-03-04T17:58:37.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.081+0000 7f97cd1ec980 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-03-04T17:58:37.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.081+0000 7f97cd1ec980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2024-03-04T17:58:37.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.081+0000 7f96ae7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824014 cookie 94214330204000 notifier 5573 bl.length()=202 2024-03-04T17:58:37.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.081+0000 7f96ae7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2024-03-04T17:58:37.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.081+0000 7f96ae7fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-03-04T17:58:37.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.085+0000 7f97cd1ec980 10 cache put: name=default.rgw.meta+users.keys+QOXRJIHOSDFWAPOBUJVJ info.flags=0x7 2024-03-04T17:58:37.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.085+0000 7f97cd1ec980 10 moving default.rgw.meta+users.keys+QOXRJIHOSDFWAPOBUJVJ to cache LRU end 2024-03-04T17:58:37.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.085+0000 7f97cd1ec980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:QOXRJIHOSDFWAPOBUJVJ, ofs0, ns] 2024-03-04T17:58:37.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.085+0000 7f96ae7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824008 cookie 94214330201024 notifier 5573 bl.length()=190 2024-03-04T17:58:37.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.085+0000 7f96ae7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+QOXRJIHOSDFWAPOBUJVJ info.flags=0x7 2024-03-04T17:58:37.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.085+0000 7f96ae7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+QOXRJIHOSDFWAPOBUJVJ to cache LRU end 2024-03-04T17:58:37.087 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-04T17:58:37.087 INFO:teuthology.orchestra.run.smithi137.stdout: "user_id": "ragweed.client.0", 2024-03-04T17:58:37.087 INFO:teuthology.orchestra.run.smithi137.stdout: "display_name": "Mr. ragweed.client.0", 2024-03-04T17:58:37.087 INFO:teuthology.orchestra.run.smithi137.stdout: "email": "ragweed.client.0+test@test.test", 2024-03-04T17:58:37.087 INFO:teuthology.orchestra.run.smithi137.stdout: "suspended": 0, 2024-03-04T17:58:37.087 INFO:teuthology.orchestra.run.smithi137.stdout: "max_buckets": 1000, 2024-03-04T17:58:37.087 INFO:teuthology.orchestra.run.smithi137.stdout: "subusers": [], 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: "keys": [ 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: "user": "ragweed.client.0", 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: "access_key": "QOXRJIHOSDFWAPOBUJVJ", 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: "secret_key": "EGiv4KsMCxchNkaPE1FJgMpIr9berR4jiv8GJhkzQxcih+cr7hjSFA==", 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: "active": true 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: "swift_keys": [], 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": [], 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: "op_mask": "read, write, delete", 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: "default_placement": "", 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: "default_storage_class": "", 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: "placement_tags": [], 2024-03-04T17:58:37.088 INFO:teuthology.orchestra.run.smithi137.stdout: "bucket_quota": { 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: "user_quota": { 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: "temp_url_keys": [], 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "rgw", 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: "mfa_ids": [] 2024-03-04T17:58:37.089 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-04T17:58:37.090 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:58:37.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.089+0000 7f96817e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:58:37.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.093+0000 7f97cd1ec980 20 remove_watcher() i=0 2024-03-04T17:58:37.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.093+0000 7f97cd1ec980 2 removed watcher, disabling cache 2024-03-04T17:58:37.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.097+0000 7f97cd1ec980 20 remove_watcher() i=1 2024-03-04T17:58:37.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.097+0000 7f97cd1ec980 20 remove_watcher() i=2 2024-03-04T17:58:37.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.101+0000 7f97cd1ec980 20 remove_watcher() i=3 2024-03-04T17:58:37.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.105+0000 7f97cd1ec980 20 remove_watcher() i=4 2024-03-04T17:58:37.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.105+0000 7f97cd1ec980 20 remove_watcher() i=5 2024-03-04T17:58:37.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.109+0000 7f97cd1ec980 20 remove_watcher() i=6 2024-03-04T17:58:37.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.109+0000 7f97cd1ec980 20 remove_watcher() i=7 2024-03-04T17:58:37.124 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2024-03-04T17:58:37.125 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key BACTGXKRQHOLLBJINBBI --secret zZfppZHRkMaJJCpd0Z8DIGBWeMxPwp1W982P/Hz0J7lPwg7oFofRow== --email sysuser.client.0+test@test.test --system true 2024-03-04T17:58:37.167 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T17:58:37.168 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T17:58:37.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.209+0000 7f31f763c980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.209+0000 7f31f763c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.209+0000 7f31f763c980 20 realm 2024-03-04T17:58:37.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.209+0000 7f31f763c980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.209+0000 7f31f763c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.209+0000 7f31f763c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T17:58:37.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.209+0000 7f31f763c980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.209+0000 7f31f763c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.209+0000 7f31f763c980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.213+0000 7f31f763c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:58:37.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.213+0000 7f31f763c980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.213+0000 7f31f763c980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T17:58:37.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.213+0000 7f31f763c980 20 searching for the correct realm 2024-03-04T17:58:37.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.217+0000 7f31f5955640 0 -- 172.21.15.137:0/2695953267 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f31f0004040 msgr2=0x7f31f00e0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T17:58:37.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.433+0000 7f31f5955640 0 -- 172.21.15.137:0/2695953267 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55dc71794240 msgr2=0x55dc71796700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T17:58:37.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.441+0000 7f31f763c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T17:58:37.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.441+0000 7f31f763c980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T17:58:37.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.441+0000 7f31f763c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T17:58:37.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.441+0000 7f31f763c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T17:58:37.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.441+0000 7f31f763c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T17:58:37.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.441+0000 7f31f763c980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T17:58:37.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.441+0000 7f31f763c980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T17:58:37.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T17:58:37.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 20 zone default found 2024-03-04T17:58:37.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 4 Realm: () 2024-03-04T17:58:37.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T17:58:37.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T17:58:37.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T17:58:37.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 20 zonegroup default 2024-03-04T17:58:37.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.445+0000 7f31f763c980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.449+0000 7f31f763c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.449+0000 7f31f763c980 20 started sync module instance, tier type = 2024-03-04T17:58:37.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.449+0000 7f31f763c980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T17:58:37.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.469+0000 7f31f763c980 20 add_watcher() i=0 2024-03-04T17:58:37.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.469+0000 7f31f763c980 20 add_watcher() i=1 2024-03-04T17:58:37.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.469+0000 7f31f763c980 20 add_watcher() i=2 2024-03-04T17:58:37.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.469+0000 7f31f763c980 20 add_watcher() i=3 2024-03-04T17:58:37.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.469+0000 7f31f763c980 20 add_watcher() i=4 2024-03-04T17:58:37.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.469+0000 7f31f763c980 20 add_watcher() i=5 2024-03-04T17:58:37.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.469+0000 7f31f763c980 20 add_watcher() i=6 2024-03-04T17:58:37.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.473+0000 7f31f763c980 20 add_watcher() i=7 2024-03-04T17:58:37.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.473+0000 7f31f763c980 2 all 8 watchers are set, enabling cache 2024-03-04T17:58:37.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.477+0000 7f30ab7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:58:37.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.477+0000 7f31f763c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T17:58:37.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.477+0000 7f31f763c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T17:58:37.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.477+0000 7f31f763c980 5 note: GC not initialized 2024-03-04T17:58:37.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.477+0000 7f30aa7e4640 20 reqs_thread_entry: start 2024-03-04T17:58:37.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.477+0000 7f31f763c980 20 init_complete bucket index max shards: 1 2024-03-04T17:58:37.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.477+0000 7f31f763c980 20 Filter name: none 2024-03-04T17:58:37.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.477+0000 7f301bfff640 20 reqs_thread_entry: start 2024-03-04T17:58:37.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.477+0000 7f31f763c980 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2024-03-04T17:58:37.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.477+0000 7f31f763c980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2024-03-04T17:58:37.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-03-04T17:58:37.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2024-03-04T17:58:37.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2024-03-04T17:58:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-03-04T17:58:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 10 cache get: name=default.rgw.meta+users.keys+BACTGXKRQHOLLBJINBBI : miss 2024-03-04T17:58:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 20 rados->read ofs=0 len=0 2024-03-04T17:58:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T17:58:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 10 cache put: name=default.rgw.meta+users.keys+BACTGXKRQHOLLBJINBBI info.flags=0x0 2024-03-04T17:58:37.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 10 adding default.rgw.meta+users.keys+BACTGXKRQHOLLBJINBBI to cache LRU end 2024-03-04T17:58:37.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 10 cache get: name=default.rgw.meta+users.keys+BACTGXKRQHOLLBJINBBI : hit (negative entry) 2024-03-04T17:58:37.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.481+0000 7f31f763c980 10 cache get: name=default.rgw.meta+users.keys+BACTGXKRQHOLLBJINBBI : hit (negative entry) 2024-03-04T17:58:37.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.485+0000 7f31f763c980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2024-03-04T17:58:37.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.485+0000 7f31f763c980 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-03-04T17:58:37.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.485+0000 7f31f763c980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2024-03-04T17:58:37.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.485+0000 7f30d8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824015 cookie 94405284955728 notifier 5853 bl.length()=642 2024-03-04T17:58:37.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.485+0000 7f30d8ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2024-03-04T17:58:37.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.485+0000 7f30d8ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-03-04T17:58:37.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.489+0000 7f31f763c980 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2024-03-04T17:58:37.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.489+0000 7f31f763c980 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-03-04T17:58:37.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.489+0000 7f31f763c980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2024-03-04T17:58:37.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.489+0000 7f30d8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824069 cookie 94405284961552 notifier 5853 bl.length()=202 2024-03-04T17:58:37.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.489+0000 7f30d8ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2024-03-04T17:58:37.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.489+0000 7f30d8ff9640 10 rgw watcher librados: moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-03-04T17:58:37.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.493+0000 7f31f763c980 10 cache put: name=default.rgw.meta+users.keys+BACTGXKRQHOLLBJINBBI info.flags=0x7 2024-03-04T17:58:37.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.493+0000 7f31f763c980 10 moving default.rgw.meta+users.keys+BACTGXKRQHOLLBJINBBI to cache LRU end 2024-03-04T17:58:37.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.493+0000 7f31f763c980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.keys:BACTGXKRQHOLLBJINBBI, ofs0, ns] 2024-03-04T17:58:37.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.493+0000 7f30c7fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824011 cookie 94405284968736 notifier 5853 bl.length()=190 2024-03-04T17:58:37.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.493+0000 7f30c7fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+BACTGXKRQHOLLBJINBBI info.flags=0x7 2024-03-04T17:58:37.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.493+0000 7f30c7fff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+BACTGXKRQHOLLBJINBBI to cache LRU end 2024-03-04T17:58:37.494 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-04T17:58:37.494 INFO:teuthology.orchestra.run.smithi137.stdout: "user_id": "sysuser.client.0", 2024-03-04T17:58:37.494 INFO:teuthology.orchestra.run.smithi137.stdout: "display_name": "Mr. sysuser.client.0", 2024-03-04T17:58:37.494 INFO:teuthology.orchestra.run.smithi137.stdout: "email": "sysuser.client.0+test@test.test", 2024-03-04T17:58:37.494 INFO:teuthology.orchestra.run.smithi137.stdout: "suspended": 0, 2024-03-04T17:58:37.494 INFO:teuthology.orchestra.run.smithi137.stdout: "max_buckets": 1000, 2024-03-04T17:58:37.494 INFO:teuthology.orchestra.run.smithi137.stdout: "subusers": [], 2024-03-04T17:58:37.494 INFO:teuthology.orchestra.run.smithi137.stdout: "keys": [ 2024-03-04T17:58:37.494 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T17:58:37.494 INFO:teuthology.orchestra.run.smithi137.stdout: "user": "sysuser.client.0", 2024-03-04T17:58:37.494 INFO:teuthology.orchestra.run.smithi137.stdout: "access_key": "BACTGXKRQHOLLBJINBBI", 2024-03-04T17:58:37.494 INFO:teuthology.orchestra.run.smithi137.stdout: "secret_key": "zZfppZHRkMaJJCpd0Z8DIGBWeMxPwp1W982P/Hz0J7lPwg7oFofRow==", 2024-03-04T17:58:37.494 INFO:teuthology.orchestra.run.smithi137.stdout: "active": true 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: "swift_keys": [], 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": [], 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: "op_mask": "read, write, delete", 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: "system": true, 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: "default_placement": "", 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: "default_storage_class": "", 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: "placement_tags": [], 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: "bucket_quota": { 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T17:58:37.495 INFO:teuthology.orchestra.run.smithi137.stdout: "user_quota": { 2024-03-04T17:58:37.496 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T17:58:37.496 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T17:58:37.496 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T17:58:37.496 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T17:58:37.496 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T17:58:37.496 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T17:58:37.496 INFO:teuthology.orchestra.run.smithi137.stdout: "temp_url_keys": [], 2024-03-04T17:58:37.496 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "rgw", 2024-03-04T17:58:37.496 INFO:teuthology.orchestra.run.smithi137.stdout: "mfa_ids": [] 2024-03-04T17:58:37.496 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-04T17:58:37.496 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:58:37.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.497+0000 7f30ab7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T17:58:37.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.501+0000 7f31f763c980 20 remove_watcher() i=0 2024-03-04T17:58:37.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.501+0000 7f31f763c980 2 removed watcher, disabling cache 2024-03-04T17:58:37.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.505+0000 7f31f763c980 20 remove_watcher() i=1 2024-03-04T17:58:37.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.505+0000 7f31f763c980 20 remove_watcher() i=2 2024-03-04T17:58:37.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.509+0000 7f31f763c980 20 remove_watcher() i=3 2024-03-04T17:58:37.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.509+0000 7f31f763c980 20 remove_watcher() i=4 2024-03-04T17:58:37.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.513+0000 7f31f763c980 20 remove_watcher() i=5 2024-03-04T17:58:37.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.517+0000 7f31f763c980 20 remove_watcher() i=6 2024-03-04T17:58:37.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T17:58:37.517+0000 7f31f763c980 20 remove_watcher() i=7 2024-03-04T17:58:37.533 INFO:tasks.ragweed:Configuring ragweed... 2024-03-04T17:58:37.533 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:58:37.533 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/archive/ragweed.client.0.conf 2024-03-04T17:58:37.578 INFO:tasks.ragweed:Configuring boto... 2024-03-04T17:58:37.579 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T17:58:37.579 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/boto.cfg 2024-03-04T17:58:37.626 DEBUG:teuthology.orchestra.run.smithi137:ragweed tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw' 2024-03-04T17:58:38.049 INFO:teuthology.orchestra.run.smithi137.stdout:GLOB sdist-make: /home/ubuntu/cephtest/ragweed.client.0/setup.py 2024-03-04T17:58:38.250 INFO:teuthology.orchestra.run.smithi137.stdout:py create: /home/ubuntu/cephtest/ragweed.client.0/.tox/py 2024-03-04T17:58:38.752 INFO:teuthology.orchestra.run.smithi137.stdout:py installdeps: -rrequirements.txt 2024-03-04T17:58:42.246 INFO:teuthology.orchestra.run.smithi137.stdout:py inst: /home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.zip 2024-03-04T17:58:44.973 INFO:teuthology.orchestra.run.smithi137.stdout:py installed: appdirs==1.4.4,asyncssh==2.5.0,attrs==21.2.0,Babel==2.8.0,bcrypt==3.2.0,beautifulsoup4==4.10.0,beniget==0.4.1,blinker==1.4,boto==2.49.0,Brotli==1.0.9,cachetools==5.0.0,ceph==1.0.0,ceph-volume==1.0.0,cephfs==2.0.0,certifi==2020.6.20,chardet==4.0.0,cheroot==8.5.2+ds1,CherryPy==18.6.1,command-not-found==0.3,configobj==5.0.6,cryptography==3.4.8,cycler==0.11.0,dbus-python==1.2.18,decorator==4.4.2,distlib==0.3.4,distro==1.7.0,distro-info===1.1build1,docopt==0.6.2,execnet==1.9.0,filelock==3.6.0,fonttools==4.29.1,fs==2.4.12,gast==0.5.2,gevent==21.8.0,google-auth==1.5.1,greenlet==1.1.2,html5lib==1.1,httplib2==0.20.2,idna==3.3,importlib-metadata==4.6.4,iniconfig==1.1.1,isodate==0.6.1,jaraco.classes==3.2.1,jaraco.collections==3.4.0,jaraco.functools==3.4.0,jaraco.text==3.6.0,jeepney==0.7.1,Jinja2==3.0.3,joblib==0.17.0,keyring==23.5.0,kiwisolver==1.3.2,kubernetes==12.0.1,language-selector==0.1,launchpadlib==1.10.16,lazr.restfulclient==0.14.4,lazr.uri==1.0.6,logutils==0.3.3,lxml==4.8.0,lz4==3.1.3+dfsg,Mako==1.1.3,MarkupSafe==2.0.1,matplotlib==3.5.1,more-itertools==8.10.0,mpmath==0.0.0,munch==4.0.0,natsort==8.0.2,netifaces==0.11.0,nose==1.3.7,numpy==1.21.5,oauthlib==3.2.0,olefile==0.46,packaging==21.3,Paste==3.5.0,PasteDeploy==2.1.1,PasteScript==2.0.2,pecan==1.3.3,Pillow==9.0.1,platformdirs==2.5.1,pluggy==0.13.0,ply==3.11,portend==3.0.0,prettytable==2.5.0,psutil==5.9.0,py==1.10.0,pyasn1==0.4.8,pyasn1-modules==0.2.1,Pygments==2.11.2,PyGObject==3.42.1,pyinotify==0.9.6,PyJWT==2.3.0,pymacaroons==0.13.0,PyNaCl==1.5.0,pyOpenSSL==21.0.0,pyparsing==2.4.7,pytest==6.2.5,python-apt==2.3.0+ubuntu2.1,python-dateutil==2.8.1,python-magic==0.4.24,pythran==0.10.0,pytz==2022.1,PyYAML==5.4.1,rados==2.0.0,ragweed @ file:///home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.zip#sha256=83abc0d02e97658626e6711e98f6251cb2dd1fb888e1988414de504fedc62489,rbd==2.0.0,registries-conf-ctl @ git+https://github.com/sebastian-philipp/registries-conf-ctl@0e9bf2dc202aa9be9b0bf8e8265aac3f2308f9db,remoto==1.2.1,repoze.lru==0.7,requests==2.25.1,requests-oauthlib==1.3.0,rgw==2.0.0,Routes==2.5.1,rsa==4.8,s3cmd==2.2.0,scikit-learn==0.23.2,scipy==1.8.0,SecretStorage==3.3.1,simplegeneric==0.8.1,simplejson==3.17.6,six==1.16.0,soupsieve==2.3.1,ssh-import-id==5.11,sympy==1.9,Tempita==0.5.2,tempora==4.1.2,threadpoolctl==3.1.0,toml==0.10.2,ubuntu-advantage-tools==27.12,ufoLib2==0.13.1,ufw==0.36.1,unattended-upgrades==0.1,unicodedata2==14.0.0,urllib3==1.26.5,virtualenv==20.13.0+ds,wadllib==1.3.6,waitress==1.4.4,wcwidth==0.2.5,webencodings==0.5.1,WebOb==1.8.6,websocket-client==1.2.3,WebTest==2.0.35,Werkzeug==2.0.2,zc.lockfile==2.0,zipp==1.0.0,zope.event==4.4,zope.interface==5.4.0 2024-03-04T17:58:44.973 INFO:teuthology.orchestra.run.smithi137.stdout:py run-test-pre: PYTHONHASHSEED='1762435081' 2024-03-04T17:58:44.973 INFO:teuthology.orchestra.run.smithi137.stdout:py run-test: commands[0] | python -m pytest -v -m 'not fails_on_rgw' 2024-03-04T17:58:45.138 INFO:teuthology.orchestra.run.smithi137.stdout:============================= test session starts ============================== 2024-03-04T17:58:45.138 INFO:teuthology.orchestra.run.smithi137.stdout:platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0 -- /home/ubuntu/cephtest/ragweed.client.0/.tox/py/bin/python 2024-03-04T17:58:45.138 INFO:teuthology.orchestra.run.smithi137.stdout:cachedir: .tox/py/.pytest_cache 2024-03-04T17:58:45.138 INFO:teuthology.orchestra.run.smithi137.stdout:rootdir: /home/ubuntu/cephtest/ragweed.client.0, configfile: pytest.ini 2024-03-04T17:58:45.297 INFO:teuthology.orchestra.run.smithi137.stdout:collecting ... collected 9 items 2024-03-04T17:58:45.297 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T17:58:56.437 INFO:teuthology.orchestra.run.smithi137.stdout:ragweed/tests/tests.py::r_test_small_obj_data::test PASSED [ 11%] 2024-03-04T17:59:10.117 INFO:teuthology.orchestra.run.smithi137.stdout:ragweed/tests/tests.py::r_test_multipart_simple::test PASSED [ 22%] 2024-03-04T17:59:22.118 INFO:teuthology.orchestra.run.smithi137.stdout:ragweed/tests/tests.py::r_test_multipart_index_versioning::test PASSED [ 33%] 2024-03-04T17:59:36.511 INFO:teuthology.orchestra.run.smithi137.stdout:ragweed/tests/tests.py::r_test_multipart_defer_complete::test PASSED [ 44%] 2024-03-04T17:59:49.161 INFO:teuthology.orchestra.run.smithi137.stdout:ragweed/tests/tests.py::r_test_multipart_defer_update_complete::test PASSED [ 55%] 2024-03-04T18:00:14.525 INFO:teuthology.orchestra.run.smithi137.stdout:ragweed/tests/tests.py::r_test_obj_storage_class::test PASSED [ 66%] 2024-03-04T18:00:40.687 INFO:teuthology.orchestra.run.smithi137.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart::test PASSED [ 77%] 2024-03-04T18:01:05.061 INFO:teuthology.orchestra.run.smithi137.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_copy::test PASSED [ 88%] 2024-03-04T18:01:51.914 INFO:teuthology.orchestra.run.smithi137.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart_copy::test PASSED [100%] 2024-03-04T18:01:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:01:51.914 INFO:teuthology.orchestra.run.smithi137.stdout:=============================== warnings summary =============================== 2024-03-04T18:01:51.914 INFO:teuthology.orchestra.run.smithi137.stdout:.tox/py/lib/python3.10/site-packages/boto/plugin.py:40 2024-03-04T18:01:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: /home/ubuntu/cephtest/ragweed.client.0/.tox/py/lib/python3.10/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses 2024-03-04T18:01:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: import imp 2024-03-04T18:01:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:01:51.914 INFO:teuthology.orchestra.run.smithi137.stdout:-- Docs: https://docs.pytest.org/en/stable/warnings.html 2024-03-04T18:01:51.915 INFO:teuthology.orchestra.run.smithi137.stdout:=================== 9 passed, 1 warning in 186.77s (0:03:06) =================== 2024-03-04T18:01:51.969 INFO:teuthology.orchestra.run.smithi137.stdout:___________________________________ summary ____________________________________ 2024-03-04T18:01:51.969 INFO:teuthology.orchestra.run.smithi137.stdout: py: commands succeeded 2024-03-04T18:01:51.970 INFO:teuthology.orchestra.run.smithi137.stdout: congratulations :) 2024-03-04T18:01:51.987 INFO:tasks.ragweed:Cleaning up boto... 2024-03-04T18:01:51.987 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/boto.cfg 2024-03-04T18:01:51.993 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2024-03-04T18:01:52.070 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:01:52.070 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:01:52.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.108+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.108+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:52.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.108+0000 7f3f691dc980 20 realm 2024-03-04T18:01:52.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.108+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.108+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.108+0000 7f3f691dc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:01:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.108+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.108+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.108+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.112+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:01:52.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.112+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.112+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:01:52.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.112+0000 7f3f691dc980 20 searching for the correct realm 2024-03-04T18:01:52.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.116+0000 7f3f674f5640 0 -- 172.21.15.137:0/3293830205 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f3f6005be10 msgr2=0x7f3f6006b0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:01:52.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.332+0000 7f3f674f5640 0 -- 172.21.15.137:0/3293830205 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55af886f4a00 msgr2=0x55af886f6ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:01:52.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.340+0000 7f3f691dc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:01:52.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.340+0000 7f3f691dc980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:01:52.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.340+0000 7f3f691dc980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:01:52.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.340+0000 7f3f691dc980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:01:52.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.340+0000 7f3f691dc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:01:52.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.340+0000 7f3f691dc980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:01:52.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.340+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.340+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:52.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.340+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:01:52.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:01:52.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 20 zone default found 2024-03-04T18:01:52.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 4 Realm: () 2024-03-04T18:01:52.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:01:52.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:01:52.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:01:52.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 20 zonegroup default 2024-03-04T18:01:52.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:52.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:52.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:52.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 20 started sync module instance, tier type = 2024-03-04T18:01:52.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.344+0000 7f3f691dc980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:01:52.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.368+0000 7f3f691dc980 20 add_watcher() i=0 2024-03-04T18:01:52.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.368+0000 7f3f691dc980 20 add_watcher() i=1 2024-03-04T18:01:52.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.368+0000 7f3f691dc980 20 add_watcher() i=2 2024-03-04T18:01:52.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.368+0000 7f3f691dc980 20 add_watcher() i=3 2024-03-04T18:01:52.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.368+0000 7f3f691dc980 20 add_watcher() i=4 2024-03-04T18:01:52.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.368+0000 7f3f691dc980 20 add_watcher() i=5 2024-03-04T18:01:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.368+0000 7f3f691dc980 20 add_watcher() i=6 2024-03-04T18:01:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.372+0000 7f3f691dc980 20 add_watcher() i=7 2024-03-04T18:01:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.372+0000 7f3f691dc980 2 all 8 watchers are set, enabling cache 2024-03-04T18:01:52.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.376+0000 7f3e1d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:01:52.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.376+0000 7f3f691dc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:01:52.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.376+0000 7f3f691dc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:01:52.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.468+0000 7f3d97fff640 20 reqs_thread_entry: start 2024-03-04T18:01:52.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.468+0000 7f3f691dc980 20 init_complete bucket index max shards: 1 2024-03-04T18:01:52.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.468+0000 7f3f691dc980 20 Filter name: none 2024-03-04T18:01:52.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.468+0000 7f3d95ffb640 20 reqs_thread_entry: start 2024-03-04T18:01:52.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.468+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2024-03-04T18:01:52.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.468+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.468+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=453 2024-03-04T18:01:52.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.468+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x13 2024-03-04T18:01:52.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.468+0000 7f3f691dc980 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-03-04T18:01:52.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.468+0000 7f3f691dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:01:52.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5 : miss 2024-03-04T18:01:52.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=378 2024-03-04T18:01:52.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5 info.flags=0x17 2024-03-04T18:01:52.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5 to cache LRU end 2024-03-04T18:01:52.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:01:52.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:01:52.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5 2024-03-04T18:01:52.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete 2024-03-04T18:01:52.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete 2024-03-04T18:01:52.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning 2024-03-04T18:01:52.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 cls_bucket_list_unordered: got tests/r_test_multipart_simple 2024-03-04T18:01:52.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class 2024-03-04T18:01:52.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy 2024-03-04T18:01:52.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart 2024-03-04T18:01:52.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy 2024-03-04T18:01:52.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 cls_bucket_list_unordered: got tests/r_test_small_obj_data 2024-03-04T18:01:52.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5805f2b0 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x55af887a3f18 s->prefetch_data=0 2024-03-04T18:01:52.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 10 manifest: total_size = 5146 2024-03-04T18:01:52.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8020718763462565440 2024-03-04T18:01:52.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5805f2b0 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x55af887a3f18 s->prefetch_data=0 2024-03-04T18:01:52.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af887a3f18 2024-03-04T18:01:52.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.472+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5.0 2024-03-04T18:01:52.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.476+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5805f2b0 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x55af8872f3d8 s->prefetch_data=0 2024-03-04T18:01:52.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.476+0000 7f3f691dc980 10 manifest: total_size = 5149 2024-03-04T18:01:52.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.476+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16663231637176273817 2024-03-04T18:01:52.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.476+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5805f2b0 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x55af8872f3d8 s->prefetch_data=0 2024-03-04T18:01:52.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.476+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8872f3d8 2024-03-04T18:01:52.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.476+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5.0 2024-03-04T18:01:52.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.480+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_complete 2024-03-04T18:01:52.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.480+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5805f2b0 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x55af8872f3d8 s->prefetch_data=0 2024-03-04T18:01:52.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.480+0000 7f3f691dc980 10 manifest: total_size = 7400 2024-03-04T18:01:52.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.480+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7738497400000175711 2024-03-04T18:01:52.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.480+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5805f2b0 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x55af8872f3d8 s->prefetch_data=0 2024-03-04T18:01:52.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.480+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8872f3d8 2024-03-04T18:01:52.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.480+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5.0 2024-03-04T18:01:52.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.484+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_update_complete 2024-03-04T18:01:52.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.484+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5805f2b0 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x55af8872f3d8 s->prefetch_data=0 2024-03-04T18:01:52.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.488+0000 7f3f691dc980 10 manifest: total_size = 4822 2024-03-04T18:01:52.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.488+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2633952236447393523 2024-03-04T18:01:52.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.488+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5805f2b0 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x55af8872f3d8 s->prefetch_data=0 2024-03-04T18:01:52.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.488+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8872f3d8 2024-03-04T18:01:52.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.488+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5.0 2024-03-04T18:01:52.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.488+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_index_versioning 2024-03-04T18:01:52.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.488+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5805f2b0 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x55af8872f3d8 s->prefetch_data=0 2024-03-04T18:01:52.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.492+0000 7f3f691dc980 10 manifest: total_size = 7436 2024-03-04T18:01:52.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.492+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.3370565632430875901 2024-03-04T18:01:52.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.492+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5805f2b0 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x55af8872f3d8 s->prefetch_data=0 2024-03-04T18:01:52.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.492+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8872f3d8 2024-03-04T18:01:52.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.492+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5.0 2024-03-04T18:01:52.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.492+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_simple 2024-03-04T18:01:52.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.496+0000 7f3f691dc980 20 get_obj_state: octx=0x55af887290a0 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x55af8872f3d8 s->prefetch_data=0 2024-03-04T18:01:52.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.496+0000 7f3f691dc980 10 manifest: total_size = 7587 2024-03-04T18:01:52.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.496+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.3933134405362740404 2024-03-04T18:01:52.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.496+0000 7f3f691dc980 20 get_obj_state: octx=0x55af887290a0 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x55af8872f3d8 s->prefetch_data=0 2024-03-04T18:01:52.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.496+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8872f3d8 2024-03-04T18:01:52.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.496+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5.0 2024-03-04T18:01:52.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.496+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class 2024-03-04T18:01:52.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.500+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5805f2b0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x55af8872f3d8 s->prefetch_data=0 2024-03-04T18:01:52.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.500+0000 7f3f691dc980 10 manifest: total_size = 5317 2024-03-04T18:01:52.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.500+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.3022653497132466808 2024-03-04T18:01:52.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.500+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5805f2b0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x55af8872f3d8 s->prefetch_data=0 2024-03-04T18:01:52.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.500+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8872f3d8 2024-03-04T18:01:52.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.500+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5.0 2024-03-04T18:01:52.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.500+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_copy 2024-03-04T18:01:52.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.504+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8872e680 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x55af8872a548 s->prefetch_data=0 2024-03-04T18:01:52.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.504+0000 7f3f691dc980 10 manifest: total_size = 7708 2024-03-04T18:01:52.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.504+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12972961951476498011 2024-03-04T18:01:52.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.504+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8872e680 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x55af8872a548 s->prefetch_data=0 2024-03-04T18:01:52.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.504+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8872a548 2024-03-04T18:01:52.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.504+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5.0 2024-03-04T18:01:52.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.504+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart 2024-03-04T18:01:52.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.508+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8872d860 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x55af8872a548 s->prefetch_data=0 2024-03-04T18:01:52.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.508+0000 7f3f691dc980 10 manifest: total_size = 8625 2024-03-04T18:01:52.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.508+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10010678503894862598 2024-03-04T18:01:52.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.508+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8872d860 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x55af8872a548 s->prefetch_data=0 2024-03-04T18:01:52.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.508+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8872a548 2024-03-04T18:01:52.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.508+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5.0 2024-03-04T18:01:52.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.512+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart_copy 2024-03-04T18:01:52.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.516+0000 7f3f691dc980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:01:52.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.516+0000 7f3f691dc980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-03-04T18:01:52.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.516+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=tests/r_test_small_obj_data 2024-03-04T18:01:52.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.516+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:01:52.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.516+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:01:52.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.516+0000 7f3f691dc980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:01:52.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.516+0000 7f3f691dc980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:01:52.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.516+0000 7f3f691dc980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5])"; uploads.size()=0; is_truncated=0 2024-03-04T18:01:52.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.520+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2024-03-04T18:01:52.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.520+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.520+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=204 2024-03-04T18:01:52.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.520+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x11 2024-03-04T18:01:52.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.520+0000 7f3f691dc980 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2024-03-04T18:01:52.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.524+0000 7f3f691dc980 10 removing default.rgw.meta+root+test-client.0-conf from cache 2024-03-04T18:01:52.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.524+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-conf, ofs0, ns] 2024-03-04T18:01:52.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.524+0000 7f3e49ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627836 cookie 94212396638336 notifier 5889 bl.length()=162 2024-03-04T18:01:52.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.528+0000 7f3f691dc980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5 from cache 2024-03-04T18:01:52.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.528+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-conf:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5, ofs0, ns] 2024-03-04T18:01:52.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.532+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627795 cookie 94212396663456 notifier 5889 bl.length()=219 2024-03-04T18:01:52.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.536+0000 7f3f691dc980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-conf/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5 : miss 2024-03-04T18:01:52.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.536+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.536+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:52.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.536+0000 7f3f691dc980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-conf/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5 info.flags=0x0 2024-03-04T18:01:52.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.536+0000 7f3f691dc980 10 adding default.rgw.log++pubsub..bucket.test-client.0-conf/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5 to cache LRU end 2024-03-04T18:01:52.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12 : miss 2024-03-04T18:01:52.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=409 2024-03-04T18:01:52.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12 info.flags=0x17 2024-03-04T18:01:52.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12 to cache LRU end 2024-03-04T18:01:52.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:01:52.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:01:52.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12 2024-03-04T18:01:52.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo 2024-03-04T18:01:52.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60053350 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x55af887294c8 s->prefetch_data=0 2024-03-04T18:01:52.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 10 manifest: total_size = 26078 2024-03-04T18:01:52.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.3199931641681391603 2024-03-04T18:01:52.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60053350 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x55af887294c8 s->prefetch_data=0 2024-03-04T18:01:52.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af887294c8 2024-03-04T18:01:52.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.544+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12.0 2024-03-04T18:01:52.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=7671 rules.size()=4 2024-03-04T18:01:52.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=7671 2024-03-04T18:01:52.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=7671 stripe_ofs=7671 part_ofs=7671 rule->part_size=7666 2024-03-04T18:01:52.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=7666 rules.size()=4 2024-03-04T18:01:52.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4201975 part_ofs=7671 rule->part_size=7666 2024-03-04T18:01:52.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=15337 stripe_ofs=15337 part_ofs=15337 rule->part_size=7674 2024-03-04T18:01:52.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=7674 rules.size()=4 2024-03-04T18:01:52.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4209641 part_ofs=15337 rule->part_size=7674 2024-03-04T18:01:52.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=23011 stripe_ofs=23011 part_ofs=23011 rule->part_size=3067 2024-03-04T18:01:52.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=3067 rules.size()=4 2024-03-04T18:01:52.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4217315 part_ofs=23011 rule->part_size=3067 2024-03-04T18:01:52.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=26078 stripe_ofs=26078 part_ofs=26078 rule->part_size=3067 2024-03-04T18:01:52.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.3199931641681391603 2024-03-04T18:01:52.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12__multipart_foo.2~R7XuWjpGEIqxR6TM1E-GwEOzOKD1pdy.1 2024-03-04T18:01:52.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 346 2024-03-04T18:01:52.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12__multipart_foo.2~R7XuWjpGEIqxR6TM1E-GwEOzOKD1pdy.2 2024-03-04T18:01:52.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 592 2024-03-04T18:01:52.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12__multipart_foo.2~R7XuWjpGEIqxR6TM1E-GwEOzOKD1pdy.3 2024-03-04T18:01:52.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 838 2024-03-04T18:01:52.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12__multipart_foo.2~R7XuWjpGEIqxR6TM1E-GwEOzOKD1pdy.4 2024-03-04T18:01:52.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1084 2024-03-04T18:01:52.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.552+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.3199931641681391603 2024-03-04T18:01:52.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.556+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo 2024-03-04T18:01:52.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.556+0000 7f3f691dc980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:01:52.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.556+0000 7f3f691dc980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-03-04T18:01:52.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.556+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:01:52.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.556+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:01:52.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.556+0000 7f3f691dc980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:01:52.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.556+0000 7f3f691dc980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:01:52.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.556+0000 7f3f691dc980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12])"; uploads.size()=0; is_truncated=0 2024-03-04T18:01:52.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.560+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2024-03-04T18:01:52.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.560+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.560+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=235 2024-03-04T18:01:52.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.560+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x11 2024-03-04T18:01:52.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.560+0000 7f3f691dc980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2024-03-04T18:01:52.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.560+0000 7f3f691dc980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2024-03-04T18:01:52.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.560+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1, ofs0, ns] 2024-03-04T18:01:52.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.564+0000 7f3e49ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627853 cookie 94212396646864 notifier 5889 bl.length()=191 2024-03-04T18:01:52.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.568+0000 7f3f691dc980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12 from cache 2024-03-04T18:01:52.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.568+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12, ofs0, ns] 2024-03-04T18:01:52.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.568+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627796 cookie 94212396663456 notifier 5889 bl.length()=249 2024-03-04T18:01:52.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.572+0000 7f3f691dc980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12 : miss 2024-03-04T18:01:52.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.572+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.572+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:52.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.572+0000 7f3f691dc980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12 info.flags=0x0 2024-03-04T18:01:52.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.572+0000 7f3f691dc980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12 to cache LRU end 2024-03-04T18:01:52.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13 : miss 2024-03-04T18:01:52.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=416 2024-03-04T18:01:52.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13 info.flags=0x17 2024-03-04T18:01:52.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13 to cache LRU end 2024-03-04T18:01:52.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:01:52.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:01:52.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13 2024-03-04T18:01:52.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo 2024-03-04T18:01:52.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88734720 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x55af887294c8 s->prefetch_data=0 2024-03-04T18:01:52.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 10 manifest: total_size = 886297 2024-03-04T18:01:52.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.874685326992247852 2024-03-04T18:01:52.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88734720 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x55af887294c8 s->prefetch_data=0 2024-03-04T18:01:52.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af887294c8 2024-03-04T18:01:52.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.580+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13.0 2024-03-04T18:01:52.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2024-03-04T18:01:52.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=246247 2024-03-04T18:01:52.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2024-03-04T18:01:52.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2024-03-04T18:01:52.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4440551 part_ofs=246247 rule->part_size=246247 2024-03-04T18:01:52.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=246247 2024-03-04T18:01:52.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2024-03-04T18:01:52.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4686798 part_ofs=492494 rule->part_size=246247 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=738741 stripe_ofs=738741 part_ofs=738741 rule->part_size=147556 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=147556 rules.size()=2 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4933045 part_ofs=738741 rule->part_size=147556 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=886297 stripe_ofs=886297 part_ofs=886297 rule->part_size=147556 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.874685326992247852 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13__multipart_foo.2~r9w9SisoYP9sAzMu8TPK7loFOqHcPLA.1 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 345 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13__multipart_foo.2~r9w9SisoYP9sAzMu8TPK7loFOqHcPLA.2 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 591 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13__multipart_foo.2~r9w9SisoYP9sAzMu8TPK7loFOqHcPLA.3 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 837 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13__multipart_foo.2~r9w9SisoYP9sAzMu8TPK7loFOqHcPLA.4 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1083 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.874685326992247852 2024-03-04T18:01:52.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo 2024-03-04T18:01:52.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:01:52.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.588+0000 7f3f691dc980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-03-04T18:01:52.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.592+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:01:52.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.592+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:01:52.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.592+0000 7f3f691dc980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:01:52.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.592+0000 7f3f691dc980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:01:52.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.592+0000 7f3f691dc980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13])"; uploads.size()=0; is_truncated=0 2024-03-04T18:01:52.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.596+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2024-03-04T18:01:52.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.596+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.596+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=242 2024-03-04T18:01:52.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.596+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x11 2024-03-04T18:01:52.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.596+0000 7f3f691dc980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2024-03-04T18:01:52.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.596+0000 7f3f691dc980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2024-03-04T18:01:52.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.596+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1, ofs0, ns] 2024-03-04T18:01:52.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.600+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627802 cookie 94212396651008 notifier 5889 bl.length()=198 2024-03-04T18:01:52.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.604+0000 7f3f691dc980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13 from cache 2024-03-04T18:01:52.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.604+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13, ofs0, ns] 2024-03-04T18:01:52.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.604+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627854 cookie 94212396655056 notifier 5889 bl.length()=256 2024-03-04T18:01:52.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.608+0000 7f3f691dc980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13 : miss 2024-03-04T18:01:52.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.608+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.612+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:52.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.612+0000 7f3f691dc980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13 info.flags=0x0 2024-03-04T18:01:52.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.612+0000 7f3f691dc980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13 to cache LRU end 2024-03-04T18:01:52.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.616+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11 : miss 2024-03-04T18:01:52.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.616+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.616+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=411 2024-03-04T18:01:52.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.616+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11 info.flags=0x17 2024-03-04T18:01:52.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.616+0000 7f3f691dc980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11 to cache LRU end 2024-03-04T18:01:52.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.616+0000 7f3f691dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:01:52.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.616+0000 7f3f691dc980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:01:52.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.616+0000 7f3f691dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11 2024-03-04T18:01:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.616+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo[75Co3U3fwqf-m3O5S2pr4uhqYn0diXk] 2024-03-04T18:01:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.616+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11.0 2024-03-04T18:01:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.620+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f600819c0 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x55af88734a68 s->prefetch_data=0 2024-03-04T18:01:52.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.620+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to tkg1mptm7de9msehl0hiy6rxasxwp5m6 2024-03-04T18:01:52.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.620+0000 7f3f691dc980 20 get_obj_state_impl: setting s->olh_tag to f7vydkdtfir6lzd1hi1b575lesqko33t 2024-03-04T18:01:52.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.620+0000 7f3f691dc980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=f7vydkdtfir6lzd1hi1b575lesqko33t 2024-03-04T18:01:52.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.620+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11.0 2024-03-04T18:01:52.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.624+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11.0 2024-03-04T18:01:52.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.624+0000 7f3f691dc980 20 olh_log_entry: epoch=3 op=2 key=foo[] 2024-03-04T18:01:52.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.624+0000 7f3f691dc980 20 olh_log_entry: epoch=3 op=3 key=foo[75Co3U3fwqf-m3O5S2pr4uhqYn0diXk] 2024-03-04T18:01:52.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.624+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f600819c0 obj=test-client.0-r-test-multipart-index-versioning.1:_:75Co3U3fwqf-m3O5S2pr4uhqYn0diXk_foo state=0x55af887402c8 s->prefetch_data=0 2024-03-04T18:01:52.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.628+0000 7f3f691dc980 10 manifest: total_size = 26080 2024-03-04T18:01:52.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.628+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13719652918541394405 2024-03-04T18:01:52.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.628+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f600819c0 obj=test-client.0-r-test-multipart-index-versioning.1:_:75Co3U3fwqf-m3O5S2pr4uhqYn0diXk_foo state=0x55af887402c8 s->prefetch_data=0 2024-03-04T18:01:52.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.628+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af887402c8 2024-03-04T18:01:52.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.628+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11.0 2024-03-04T18:01:52.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=7669 rules.size()=4 2024-03-04T18:01:52.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=7669 2024-03-04T18:01:52.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=7669 stripe_ofs=7669 part_ofs=7669 rule->part_size=7674 2024-03-04T18:01:52.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=7674 rules.size()=4 2024-03-04T18:01:52.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4201973 part_ofs=7669 rule->part_size=7674 2024-03-04T18:01:52.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=15343 stripe_ofs=15343 part_ofs=15343 rule->part_size=7670 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=7670 rules.size()=4 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4209647 part_ofs=15343 rule->part_size=7670 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=23013 stripe_ofs=23013 part_ofs=23013 rule->part_size=3067 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=3067 rules.size()=4 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4217317 part_ofs=23013 rule->part_size=3067 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=26080 stripe_ofs=26080 part_ofs=26080 rule->part_size=3067 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13719652918541394405 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11__multipart_foo.2~MPH5BNCPADH4YVw1XTdHNJTjFnERwPX.1 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 347 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11__multipart_foo.2~MPH5BNCPADH4YVw1XTdHNJTjFnERwPX.2 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 593 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11__multipart_foo.2~MPH5BNCPADH4YVw1XTdHNJTjFnERwPX.3 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 839 2024-03-04T18:01:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11__multipart_foo.2~MPH5BNCPADH4YVw1XTdHNJTjFnERwPX.4 2024-03-04T18:01:52.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1085 2024-03-04T18:01:52.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.632+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13719652918541394405 2024-03-04T18:01:52.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.636+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo[75Co3U3fwqf-m3O5S2pr4uhqYn0diXk] 2024-03-04T18:01:52.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.636+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f600819c0 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x55af88734a68 s->prefetch_data=0 2024-03-04T18:01:52.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.640+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11.0 2024-03-04T18:01:52.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.640+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11.0 2024-03-04T18:01:52.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.640+0000 7f3f691dc980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:01:52.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.640+0000 7f3f691dc980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-03-04T18:01:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.640+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:01:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.640+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:01:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.640+0000 7f3f691dc980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:01:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.640+0000 7f3f691dc980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:01:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.640+0000 7f3f691dc980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11])"; uploads.size()=0; is_truncated=0 2024-03-04T18:01:52.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.644+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2024-03-04T18:01:52.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.644+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.648+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=237 2024-03-04T18:01:52.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.648+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x11 2024-03-04T18:01:52.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.648+0000 7f3f691dc980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2024-03-04T18:01:52.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.648+0000 7f3f691dc980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2024-03-04T18:01:52.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.648+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1, ofs0, ns] 2024-03-04T18:01:52.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.648+0000 7f3e49ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627803 cookie 94212396651008 notifier 5889 bl.length()=193 2024-03-04T18:01:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.656+0000 7f3f691dc980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11 from cache 2024-03-04T18:01:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.656+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11, ofs0, ns] 2024-03-04T18:01:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.656+0000 7f3e49ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627804 cookie 94212396651008 notifier 5889 bl.length()=251 2024-03-04T18:01:52.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.660+0000 7f3f691dc980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11 : miss 2024-03-04T18:01:52.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.660+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.660+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.660+0000 7f3f691dc980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11 info.flags=0x0 2024-03-04T18:01:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.660+0000 7f3f691dc980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11 to cache LRU end 2024-03-04T18:01:52.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10 : miss 2024-03-04T18:01:52.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=401 2024-03-04T18:01:52.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10 info.flags=0x17 2024-03-04T18:01:52.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10 to cache LRU end 2024-03-04T18:01:52.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:01:52.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:01:52.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10 2024-03-04T18:01:52.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo 2024-03-04T18:01:52.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88725fb0 obj=test-client.0-r-test-multipart-simple.1:foo state=0x55af88734a38 s->prefetch_data=0 2024-03-04T18:01:52.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 10 manifest: total_size = 18874368 2024-03-04T18:01:52.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16904344668637114563 2024-03-04T18:01:52.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88725fb0 obj=test-client.0-r-test-multipart-simple.1:foo state=0x55af88734a38 s->prefetch_data=0 2024-03-04T18:01:52.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88734a38 2024-03-04T18:01:52.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.668+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10.0 2024-03-04T18:01:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=5242880 2024-03-04T18:01:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=5242880 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=14680064 stripe_ofs=14680064 part_ofs=10485760 rule->part_size=5242880 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=18874368 part_ofs=10485760 rule->part_size=5242880 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=15728640 stripe_ofs=15728640 part_ofs=15728640 rule->part_size=3145728 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=3145728 rules.size()=2 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=19922944 part_ofs=15728640 rule->part_size=3145728 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=18874368 stripe_ofs=18874368 part_ofs=18874368 rule->part_size=3145728 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16904344668637114563 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10__multipart_foo.2~RA4PcczPDlMtdmBtXJzZm2-Cv1sKurx.1 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 347 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10__shadow_foo.2~RA4PcczPDlMtdmBtXJzZm2-Cv1sKurx.1_1 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 591 2024-03-04T18:01:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10__multipart_foo.2~RA4PcczPDlMtdmBtXJzZm2-Cv1sKurx.2 2024-03-04T18:01:52.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 837 2024-03-04T18:01:52.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10__shadow_foo.2~RA4PcczPDlMtdmBtXJzZm2-Cv1sKurx.2_1 2024-03-04T18:01:52.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1081 2024-03-04T18:01:52.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10__multipart_foo.2~RA4PcczPDlMtdmBtXJzZm2-Cv1sKurx.3 2024-03-04T18:01:52.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1327 2024-03-04T18:01:52.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10__shadow_foo.2~RA4PcczPDlMtdmBtXJzZm2-Cv1sKurx.3_1 2024-03-04T18:01:52.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1571 2024-03-04T18:01:52.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10__multipart_foo.2~RA4PcczPDlMtdmBtXJzZm2-Cv1sKurx.4 2024-03-04T18:01:52.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1817 2024-03-04T18:01:52.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16904344668637114563 2024-03-04T18:01:52.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo 2024-03-04T18:01:52.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:01:52.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.676+0000 7f3f691dc980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-03-04T18:01:52.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.680+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:01:52.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.680+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:01:52.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.680+0000 7f3f691dc980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:01:52.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.680+0000 7f3f691dc980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:01:52.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.680+0000 7f3f691dc980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10])"; uploads.size()=0; is_truncated=0 2024-03-04T18:01:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.684+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2024-03-04T18:01:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.684+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.684+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=227 2024-03-04T18:01:52.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.684+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x11 2024-03-04T18:01:52.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.684+0000 7f3f691dc980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2024-03-04T18:01:52.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.688+0000 7f3f691dc980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2024-03-04T18:01:52.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.688+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-simple.1, ofs0, ns] 2024-03-04T18:01:52.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.688+0000 7f3e49ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627855 cookie 94212396655056 notifier 5889 bl.length()=183 2024-03-04T18:01:52.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.692+0000 7f3f691dc980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10 from cache 2024-03-04T18:01:52.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.692+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10, ofs0, ns] 2024-03-04T18:01:52.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.696+0000 7f3e49ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627837 cookie 94212396638336 notifier 5889 bl.length()=241 2024-03-04T18:01:52.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.700+0000 7f3f691dc980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10 : miss 2024-03-04T18:01:52.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.700+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.700+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:52.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.700+0000 7f3f691dc980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10 info.flags=0x0 2024-03-04T18:01:52.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.700+0000 7f3f691dc980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10 to cache LRU end 2024-03-04T18:01:52.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16 : miss 2024-03-04T18:01:52.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=407 2024-03-04T18:01:52.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16 info.flags=0x17 2024-03-04T18:01:52.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16 to cache LRU end 2024-03-04T18:01:52.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:01:52.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:01:52.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16 2024-03-04T18:01:52.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-03-04T18:01:52.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2024-03-04T18:01:52.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2024-03-04T18:01:52.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2024-03-04T18:01:52.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-03-04T18:01:52.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2024-03-04T18:01:52.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2024-03-04T18:01:52.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2024-03-04T18:01:52.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-03-04T18:01:52.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2024-03-04T18:01:52.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2024-03-04T18:01:52.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2024-03-04T18:01:52.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.708+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8851e130 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x55af88734a38 s->prefetch_data=0 2024-03-04T18:01:52.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.712+0000 7f3f691dc980 10 manifest: total_size = 5242880 2024-03-04T18:01:52.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.712+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2697193294386511849 2024-03-04T18:01:52.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.712+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8851e130 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x55af88734a38 s->prefetch_data=0 2024-03-04T18:01:52.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.712+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88734a38 2024-03-04T18:01:52.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.712+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16.0 2024-03-04T18:01:52.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.716+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.716+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.716+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.716+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.716+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2697193294386511849 2024-03-04T18:01:52.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.716+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.716+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.4jckuAtqyME-A0HXCRRmp5A1eKNUgIp_0 2024-03-04T18:01:52.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.716+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 337 2024-03-04T18:01:52.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.716+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.4jckuAtqyME-A0HXCRRmp5A1eKNUgIp_1 2024-03-04T18:01:52.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.716+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 574 2024-03-04T18:01:52.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.716+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.716+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2697193294386511849 2024-03-04T18:01:52.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.720+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8851e130 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x55af88734a38 s->prefetch_data=0 2024-03-04T18:01:52.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.720+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-03-04T18:01:52.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.720+0000 7f3f691dc980 10 manifest: total_size = 5242880 2024-03-04T18:01:52.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.720+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.720+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.720+0000 7f3f691dc980 10 generate_fake_tag new tag=f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.4jckuAtqyME-A0HXCRRmp5A1eKNUgIp_1_7fdc11563e2571435be2c59c8c380d8b 2024-03-04T18:01:52.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.720+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.4jckuAtqyME-A0HXCRRmp5A1eKNUgIp_1_7fdc11563e2571435be2c59c8c380d8b 2024-03-04T18:01:52.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.720+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8851e130 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x55af88734a38 s->prefetch_data=0 2024-03-04T18:01:52.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.720+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88734a38 2024-03-04T18:01:52.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.720+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16.0 2024-03-04T18:01:52.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.724+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.724+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.724+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.724+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.724+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.4jckuAtqyME-A0HXCRRmp5A1eKNUgIp_1_7fdc11563e2571435be2c59c8c380d8b 2024-03-04T18:01:52.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.724+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.724+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.4jckuAtqyME-A0HXCRRmp5A1eKNUgIp_0 2024-03-04T18:01:52.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.724+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 396 2024-03-04T18:01:52.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.724+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.4jckuAtqyME-A0HXCRRmp5A1eKNUgIp_1 2024-03-04T18:01:52.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.724+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 633 2024-03-04T18:01:52.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.724+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.724+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.4jckuAtqyME-A0HXCRRmp5A1eKNUgIp_1_7fdc11563e2571435be2c59c8c380d8b 2024-03-04T18:01:52.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.728+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2024-03-04T18:01:52.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.728+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8851e130 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.732+0000 7f3f691dc980 10 manifest: total_size = 5242880 2024-03-04T18:01:52.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.732+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to _qegkQBlWqlwZgv-gauKLFcpFmReMnCL 2024-03-04T18:01:52.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.732+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8851e130 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.732+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.732+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16.0 2024-03-04T18:01:52.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.736+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.736+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.736+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.736+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.736+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: _qegkQBlWqlwZgv-gauKLFcpFmReMnCL 2024-03-04T18:01:52.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.736+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.736+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RVCQ5x81jkkIXzZWFqabLMUVlfFF7s1_0 2024-03-04T18:01:52.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.736+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-03-04T18:01:52.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.736+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RVCQ5x81jkkIXzZWFqabLMUVlfFF7s1_1 2024-03-04T18:01:52.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.736+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 549 2024-03-04T18:01:52.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.736+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.736+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: _qegkQBlWqlwZgv-gauKLFcpFmReMnCL 2024-03-04T18:01:52.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.740+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2024-03-04T18:01:52.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.740+0000 7f3f691dc980 20 get_obj_state: octx=0x55af887311e0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.740+0000 7f3f691dc980 10 manifest: total_size = 5242880 2024-03-04T18:01:52.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.740+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to _90hHByb_ggy_cVbPRpMGjeMwyLzzKLs 2024-03-04T18:01:52.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.740+0000 7f3f691dc980 20 get_obj_state: octx=0x55af887311e0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.740+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.740+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16.0 2024-03-04T18:01:52.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.748+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.748+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.748+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: _90hHByb_ggy_cVbPRpMGjeMwyLzzKLs 2024-03-04T18:01:52.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.748+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.748+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.NIkysXa0IUNScOZUuV-utKeAhdzDjij_1 2024-03-04T18:01:52.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.748+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-03-04T18:01:52.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.748+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.748+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.18tag is: _90hHByb_ggy_cVbPRpMGjeMwyLzzKLs 2024-03-04T18:01:52.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.752+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2024-03-04T18:01:52.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.752+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60067620 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.756+0000 7f3f691dc980 10 manifest: total_size = 5242880 2024-03-04T18:01:52.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.756+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10989446842724053574 2024-03-04T18:01:52.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.756+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60067620 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.756+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.756+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16.0 2024-03-04T18:01:52.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.760+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.760+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.760+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.760+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.760+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10989446842724053574 2024-03-04T18:01:52.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.760+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.760+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RTqqWOMCXYph639V0D8sAmr4hsu-qUk_0 2024-03-04T18:01:52.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.760+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 340 2024-03-04T18:01:52.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.760+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RTqqWOMCXYph639V0D8sAmr4hsu-qUk_1 2024-03-04T18:01:52.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.760+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 579 2024-03-04T18:01:52.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.760+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.760+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10989446842724053574 2024-03-04T18:01:52.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.764+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-03-04T18:01:52.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.764+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0c2780 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.764+0000 7f3f691dc980 10 manifest: total_size = 5242880 2024-03-04T18:01:52.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.764+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to _rwJSNkoRMMIK1Lo9Vz5Bw20kCoZYvKE 2024-03-04T18:01:52.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.764+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0c2780 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.764+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.764+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16.0 2024-03-04T18:01:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.772+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.772+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.772+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.772+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.772+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: _rwJSNkoRMMIK1Lo9Vz5Bw20kCoZYvKE 2024-03-04T18:01:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.772+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.772+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.caDq23bEIJ1nynWJcaF8C308__7ZFv2_0 2024-03-04T18:01:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.772+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-03-04T18:01:52.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.772+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.caDq23bEIJ1nynWJcaF8C308__7ZFv2_1 2024-03-04T18:01:52.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.772+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 545 2024-03-04T18:01:52.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.772+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.772+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: _rwJSNkoRMMIK1Lo9Vz5Bw20kCoZYvKE 2024-03-04T18:01:52.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.776+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2024-03-04T18:01:52.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.776+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0dd720 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.776+0000 7f3f691dc980 10 manifest: total_size = 5242880 2024-03-04T18:01:52.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.776+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.776+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.776+0000 7f3f691dc980 10 generate_fake_tag new tag=f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RTqqWOMCXYph639V0D8sAmr4hsu-qUk_1_a4e170753976efedd5e737f2855d154b 2024-03-04T18:01:52.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.776+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RTqqWOMCXYph639V0D8sAmr4hsu-qUk_1_a4e170753976efedd5e737f2855d154b 2024-03-04T18:01:52.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.776+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0dd720 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.776+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.776+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16.0 2024-03-04T18:01:52.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.780+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.780+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.780+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.780+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.780+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RTqqWOMCXYph639V0D8sAmr4hsu-qUk_1_a4e170753976efedd5e737f2855d154b 2024-03-04T18:01:52.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.780+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.780+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RTqqWOMCXYph639V0D8sAmr4hsu-qUk_0 2024-03-04T18:01:52.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.780+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 398 2024-03-04T18:01:52.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.780+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RTqqWOMCXYph639V0D8sAmr4hsu-qUk_1 2024-03-04T18:01:52.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.780+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 637 2024-03-04T18:01:52.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.780+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.780+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.7tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RTqqWOMCXYph639V0D8sAmr4hsu-qUk_1_a4e170753976efedd5e737f2855d154b 2024-03-04T18:01:52.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.784+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0dd720 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.784+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2024-03-04T18:01:52.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.788+0000 7f3f691dc980 10 manifest: total_size = 5242880 2024-03-04T18:01:52.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.788+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to _J-K_7yuwUD2N9ieyruKnmaglaFbqRPH 2024-03-04T18:01:52.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.788+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0dd720 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.788+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.788+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16.0 2024-03-04T18:01:52.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.796+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.796+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.796+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: _J-K_7yuwUD2N9ieyruKnmaglaFbqRPH 2024-03-04T18:01:52.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.796+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.796+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.K_N1fudpCWjWZAYttGsRM9SjO805mRE_1 2024-03-04T18:01:52.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.796+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-03-04T18:01:52.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.796+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.796+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: _J-K_7yuwUD2N9ieyruKnmaglaFbqRPH 2024-03-04T18:01:52.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.800+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2024-03-04T18:01:52.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.800+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0ddf80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.800+0000 7f3f691dc980 10 manifest: total_size = 5242880 2024-03-04T18:01:52.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.800+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10976351442943365117 2024-03-04T18:01:52.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.800+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0ddf80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.800+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.800+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16.0 2024-03-04T18:01:52.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.808+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.808+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.808+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10976351442943365117 2024-03-04T18:01:52.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.808+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.808+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_._zR58I_qyqqbRWVAUP-6Er01I-tUXyu_1 2024-03-04T18:01:52.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.808+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 331 2024-03-04T18:01:52.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.808+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.808+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10976351442943365117 2024-03-04T18:01:52.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.812+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-03-04T18:01:52.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.812+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0ddf80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.812+0000 7f3f691dc980 10 manifest: total_size = 5242880 2024-03-04T18:01:52.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.812+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to _QwCP8FLLMzZYweg_DuaRZO7VcLpDbdu 2024-03-04T18:01:52.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.812+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0ddf80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.812+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.812+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16.0 2024-03-04T18:01:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.816+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.816+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.816+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.816+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.816+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: _QwCP8FLLMzZYweg_DuaRZO7VcLpDbdu 2024-03-04T18:01:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.816+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.816+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.loKTFsbdHc5Wwog7N5txaJ8ezlzUf3Z_0 2024-03-04T18:01:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.816+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-03-04T18:01:52.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.816+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.loKTFsbdHc5Wwog7N5txaJ8ezlzUf3Z_1 2024-03-04T18:01:52.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.816+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 545 2024-03-04T18:01:52.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.816+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.816+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: _QwCP8FLLMzZYweg_DuaRZO7VcLpDbdu 2024-03-04T18:01:52.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.820+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2024-03-04T18:01:52.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.820+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0c2780 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.820+0000 7f3f691dc980 10 manifest: total_size = 5242880 2024-03-04T18:01:52.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.820+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to _TxG7S7fyR9w-UlJvu2pQH_p6HWflGks 2024-03-04T18:01:52.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.820+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0c2780 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.820+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.820+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16.0 2024-03-04T18:01:52.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.828+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.828+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.828+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.828+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.828+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: _TxG7S7fyR9w-UlJvu2pQH_p6HWflGks 2024-03-04T18:01:52.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.828+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.828+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.josUa4jc9O-fQ447L6UEb10MtPW9smw_0 2024-03-04T18:01:52.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.828+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-03-04T18:01:52.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.828+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.josUa4jc9O-fQ447L6UEb10MtPW9smw_1 2024-03-04T18:01:52.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.828+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 549 2024-03-04T18:01:52.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.828+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.828+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: _TxG7S7fyR9w-UlJvu2pQH_p6HWflGks 2024-03-04T18:01:52.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.828+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2024-03-04T18:01:52.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.832+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0ddf80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.832+0000 7f3f691dc980 10 manifest: total_size = 5242880 2024-03-04T18:01:52.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.832+0000 7f3f691dc980 10 generate_fake_tag new tag=f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_._zR58I_qyqqbRWVAUP-6Er01I-tUXyu_1_ff50590991ce275e96000db28804cbd9 2024-03-04T18:01:52.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.832+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_._zR58I_qyqqbRWVAUP-6Er01I-tUXyu_1_ff50590991ce275e96000db28804cbd9 2024-03-04T18:01:52.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.832+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0ddf80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.832+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.832+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16.0 2024-03-04T18:01:52.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.840+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.840+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.840+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_._zR58I_qyqqbRWVAUP-6Er01I-tUXyu_1_ff50590991ce275e96000db28804cbd9 2024-03-04T18:01:52.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.840+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.840+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_._zR58I_qyqqbRWVAUP-6Er01I-tUXyu_1 2024-03-04T18:01:52.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.840+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 389 2024-03-04T18:01:52.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.840+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.840+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_._zR58I_qyqqbRWVAUP-6Er01I-tUXyu_1_ff50590991ce275e96000db28804cbd9 2024-03-04T18:01:52.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.844+0000 7f3f691dc980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:01:52.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.844+0000 7f3f691dc980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-03-04T18:01:52.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.844+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2024-03-04T18:01:52.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.844+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:01:52.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.844+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:01:52.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.844+0000 7f3f691dc980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:01:52.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.844+0000 7f3f691dc980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:01:52.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.844+0000 7f3f691dc980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16])"; uploads.size()=0; is_truncated=0 2024-03-04T18:01:52.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.848+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2024-03-04T18:01:52.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.848+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.848+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=233 2024-03-04T18:01:52.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.848+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x11 2024-03-04T18:01:52.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.848+0000 7f3f691dc980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2024-03-04T18:01:52.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.852+0000 7f3f691dc980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2024-03-04T18:01:52.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.852+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1, ofs0, ns] 2024-03-04T18:01:52.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.852+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627805 cookie 94212396651008 notifier 5889 bl.length()=189 2024-03-04T18:01:52.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.860+0000 7f3f691dc980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16 from cache 2024-03-04T18:01:52.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.860+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16, ofs0, ns] 2024-03-04T18:01:52.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.860+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627806 cookie 94212396651008 notifier 5889 bl.length()=247 2024-03-04T18:01:52.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.864+0000 7f3f691dc980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16 : miss 2024-03-04T18:01:52.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.864+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.864+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:52.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.864+0000 7f3f691dc980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16 info.flags=0x0 2024-03-04T18:01:52.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.864+0000 7f3f691dc980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16 to cache LRU end 2024-03-04T18:01:52.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17 : miss 2024-03-04T18:01:52.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=417 2024-03-04T18:01:52.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17 info.flags=0x17 2024-03-04T18:01:52.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17 to cache LRU end 2024-03-04T18:01:52.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:01:52.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:01:52.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17 2024-03-04T18:01:52.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-03-04T18:01:52.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2024-03-04T18:01:52.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2024-03-04T18:01:52.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2024-03-04T18:01:52.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-03-04T18:01:52.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2024-03-04T18:01:52.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2024-03-04T18:01:52.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2024-03-04T18:01:52.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-03-04T18:01:52.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2024-03-04T18:01:52.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2024-03-04T18:01:52.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2024-03-04T18:01:52.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8851c2a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 10 manifest: total_size = 11534336 2024-03-04T18:01:52.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15912843451726240125 2024-03-04T18:01:52.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8851c2a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.872+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17.0 2024-03-04T18:01:52.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15912843451726240125 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.1 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 368 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.1_1 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 633 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.2 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 900 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.2_1 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1165 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.3 2024-03-04T18:01:52.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1432 2024-03-04T18:01:52.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.880+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15912843451726240125 2024-03-04T18:01:52.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.884+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-03-04T18:01:52.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.884+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0dd720 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.884+0000 7f3f691dc980 10 manifest: total_size = 11534336 2024-03-04T18:01:52.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.884+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.884+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.884+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.884+0000 7f3f691dc980 10 generate_fake_tag new tag=f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.1_1_4867b1abbb328ab64766375e7f6d0f42 2024-03-04T18:01:52.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.884+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.1_1_4867b1abbb328ab64766375e7f6d0f42 2024-03-04T18:01:52.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.884+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f5c0dd720 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.884+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.884+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17.0 2024-03-04T18:01:52.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.1_1_4867b1abbb328ab64766375e7f6d0f42 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.1 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 440 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.1_1 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 705 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.2 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 972 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.2_1 2024-03-04T18:01:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1237 2024-03-04T18:01:52.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.3 2024-03-04T18:01:52.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1504 2024-03-04T18:01:52.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.888+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.1_1_4867b1abbb328ab64766375e7f6d0f42 2024-03-04T18:01:52.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.892+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2024-03-04T18:01:52.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.892+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8851db90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.892+0000 7f3f691dc980 10 manifest: total_size = 11534336 2024-03-04T18:01:52.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.892+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to _NlKJBoXIrQHizf0-j5ZnMyRCi36PbB5 2024-03-04T18:01:52.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.892+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8851db90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.892+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.892+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17.0 2024-03-04T18:01:52.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: _NlKJBoXIrQHizf0-j5ZnMyRCi36PbB5 2024-03-04T18:01:52.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.TSy8ST9GOrEzFO--nQ3CMdbrMD__UlR_0 2024-03-04T18:01:52.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-03-04T18:01:52.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.TSy8ST9GOrEzFO--nQ3CMdbrMD__UlR_1 2024-03-04T18:01:52.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 549 2024-03-04T18:01:52.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.TSy8ST9GOrEzFO--nQ3CMdbrMD__UlR_2 2024-03-04T18:01:52.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 788 2024-03-04T18:01:52.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.896+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _NlKJBoXIrQHizf0-j5ZnMyRCi36PbB5 2024-03-04T18:01:52.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.900+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2024-03-04T18:01:52.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.900+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8851db90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.900+0000 7f3f691dc980 10 manifest: total_size = 11534336 2024-03-04T18:01:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.900+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to _p1NtDaqrf3bTUBT6k1CC3gsPW1FuokH 2024-03-04T18:01:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.900+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8851db90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.900+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.900+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17.0 2024-03-04T18:01:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.908+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.908+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.908+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.908+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.908+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: _p1NtDaqrf3bTUBT6k1CC3gsPW1FuokH 2024-03-04T18:01:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.908+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.908+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.SROXVv7gX2el9hNB4DNUpkm2BDEkY8Y_1 2024-03-04T18:01:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.908+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-03-04T18:01:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.908+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.SROXVv7gX2el9hNB4DNUpkm2BDEkY8Y_2 2024-03-04T18:01:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.908+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-03-04T18:01:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.908+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.908+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.27tag is: _p1NtDaqrf3bTUBT6k1CC3gsPW1FuokH 2024-03-04T18:01:52.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.912+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8851c2a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.912+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2024-03-04T18:01:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.912+0000 7f3f691dc980 10 manifest: total_size = 11534336 2024-03-04T18:01:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.912+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.1042772702181890393 2024-03-04T18:01:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.912+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8851c2a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.912+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.912+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17.0 2024-03-04T18:01:52.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:01:52.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-03-04T18:01:52.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:01:52.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-03-04T18:01:52.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.1042772702181890393 2024-03-04T18:01:52.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.1 2024-03-04T18:01:52.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 373 2024-03-04T18:01:52.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.1_1 2024-03-04T18:01:52.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 644 2024-03-04T18:01:52.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.2 2024-03-04T18:01:52.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 917 2024-03-04T18:01:52.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.2_1 2024-03-04T18:01:52.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1188 2024-03-04T18:01:52.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.3 2024-03-04T18:01:52.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1461 2024-03-04T18:01:52.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.916+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.1042772702181890393 2024-03-04T18:01:52.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.920+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-03-04T18:01:52.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.920+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8872d890 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.920+0000 7f3f691dc980 10 manifest: total_size = 11534336 2024-03-04T18:01:52.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.920+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to _Z-8Jc8rUmqw3-MYkgcKZPVy3x6dz268 2024-03-04T18:01:52.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.920+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8872d890 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.920+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.920+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17.0 2024-03-04T18:01:52.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: _Z-8Jc8rUmqw3-MYkgcKZPVy3x6dz268 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.xOSVFtoJu6VjTAIM3u8UDX-vIOFPOPb_0 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.xOSVFtoJu6VjTAIM3u8UDX-vIOFPOPb_1 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 545 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.xOSVFtoJu6VjTAIM3u8UDX-vIOFPOPb_2 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 782 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.924+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.20tag is: _Z-8Jc8rUmqw3-MYkgcKZPVy3x6dz268 2024-03-04T18:01:52.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.928+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2024-03-04T18:01:52.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.928+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8872d890 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.932+0000 7f3f691dc980 10 manifest: total_size = 11534336 2024-03-04T18:01:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.932+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.932+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.932+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.932+0000 7f3f691dc980 10 generate_fake_tag new tag=f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.1_1_b4a185200b990cf923cf0a61d75c7a91 2024-03-04T18:01:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.932+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.1_1_b4a185200b990cf923cf0a61d75c7a91 2024-03-04T18:01:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.932+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8872d890 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.932+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.932+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17.0 2024-03-04T18:01:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:01:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-03-04T18:01:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.1_1_b4a185200b990cf923cf0a61d75c7a91 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.1 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 448 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.1_1 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 719 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.2 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 992 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.2_1 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1263 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.3 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1536 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.936+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.1_1_b4a185200b990cf923cf0a61d75c7a91 2024-03-04T18:01:52.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.940+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2024-03-04T18:01:52.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.940+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88741510 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.940+0000 7f3f691dc980 10 manifest: total_size = 11534336 2024-03-04T18:01:52.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.940+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to _Q3ZWNU-Pw_prsppMiTcDLtG-KotW95S 2024-03-04T18:01:52.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.940+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88741510 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.940+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.940+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17.0 2024-03-04T18:01:52.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.948+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.948+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.948+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.948+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.948+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: _Q3ZWNU-Pw_prsppMiTcDLtG-KotW95S 2024-03-04T18:01:52.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.948+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.948+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.E9aNPC3qlrcDbdCt2UPifaaDXgBq15u_1 2024-03-04T18:01:52.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.948+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-03-04T18:01:52.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.948+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.E9aNPC3qlrcDbdCt2UPifaaDXgBq15u_2 2024-03-04T18:01:52.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.948+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-03-04T18:01:52.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.948+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.948+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: _Q3ZWNU-Pw_prsppMiTcDLtG-KotW95S 2024-03-04T18:01:52.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.948+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2024-03-04T18:01:52.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.952+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88741510 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.952+0000 7f3f691dc980 10 manifest: total_size = 16378 2024-03-04T18:01:52.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.952+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11571771833839545505 2024-03-04T18:01:52.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.952+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88741510 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.952+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.952+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17.0 2024-03-04T18:01:52.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=7672 rules.size()=3 2024-03-04T18:01:52.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=7672 2024-03-04T18:01:52.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=7672 stripe_ofs=7672 part_ofs=7672 rule->part_size=7671 2024-03-04T18:01:52.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=7671 rules.size()=3 2024-03-04T18:01:52.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4201976 part_ofs=7672 rule->part_size=7671 2024-03-04T18:01:52.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=15343 stripe_ofs=15343 part_ofs=15343 rule->part_size=1035 2024-03-04T18:01:52.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=1035 rules.size()=3 2024-03-04T18:01:52.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4209647 part_ofs=15343 rule->part_size=1035 2024-03-04T18:01:52.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=16378 stripe_ofs=16378 part_ofs=16378 rule->part_size=1035 2024-03-04T18:01:52.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11571771833839545505 2024-03-04T18:01:52.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.1 2024-03-04T18:01:52.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 365 2024-03-04T18:01:52.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.2 2024-03-04T18:01:52.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 629 2024-03-04T18:01:52.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.3 2024-03-04T18:01:52.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 893 2024-03-04T18:01:52.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.956+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.7tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11571771833839545505 2024-03-04T18:01:52.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.960+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-03-04T18:01:52.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.960+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88728b50 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.960+0000 7f3f691dc980 10 manifest: total_size = 16378 2024-03-04T18:01:52.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.960+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to _w-EYfF1oQ5unXdigbGQeEIZTTcTPrk0 2024-03-04T18:01:52.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.960+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88728b50 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.960+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.960+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17.0 2024-03-04T18:01:52.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.964+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.964+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=16378 stripe_ofs=16378 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.964+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: _w-EYfF1oQ5unXdigbGQeEIZTTcTPrk0 2024-03-04T18:01:52.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.964+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.964+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.m-iJ5fVdcIxnFHP5ZgRESWTgprLMnIL_0 2024-03-04T18:01:52.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.964+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-03-04T18:01:52.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.964+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.964+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.2tag is: _w-EYfF1oQ5unXdigbGQeEIZTTcTPrk0 2024-03-04T18:01:52.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.968+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2024-03-04T18:01:52.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.968+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60052320 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.968+0000 7f3f691dc980 10 manifest: total_size = 16378 2024-03-04T18:01:52.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.968+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to _LJf1bjafIKSjGAwnWqz-qah9JjzNfwm 2024-03-04T18:01:52.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.968+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60052320 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.968+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.968+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17.0 2024-03-04T18:01:52.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.976+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:52.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.976+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=16378 stripe_ofs=16378 part_ofs=0 rule->part_size=0 2024-03-04T18:01:52.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.976+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: _LJf1bjafIKSjGAwnWqz-qah9JjzNfwm 2024-03-04T18:01:52.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.976+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.976+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.-8XtQ647UsHdVRWlV1knGZWxVbQkDhg_0 2024-03-04T18:01:52.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.976+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-03-04T18:01:52.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.976+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.976+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.7tag is: _LJf1bjafIKSjGAwnWqz-qah9JjzNfwm 2024-03-04T18:01:52.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.976+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2024-03-04T18:01:52.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.976+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f6000fff0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.980+0000 7f3f691dc980 10 manifest: total_size = 16378 2024-03-04T18:01:52.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.980+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=7672 rules.size()=3 2024-03-04T18:01:52.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.980+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=7672 2024-03-04T18:01:52.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.980+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=7672 stripe_ofs=7672 part_ofs=7672 rule->part_size=7671 2024-03-04T18:01:52.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.980+0000 7f3f691dc980 10 generate_fake_tag new tag=f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.2_e455ae1c2e5a219f513d7aaddc21adc8 2024-03-04T18:01:52.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.980+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.2_e455ae1c2e5a219f513d7aaddc21adc8 2024-03-04T18:01:52.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.980+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f6000fff0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x55af88729498 s->prefetch_data=0 2024-03-04T18:01:52.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.980+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88729498 2024-03-04T18:01:52.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.980+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17.0 2024-03-04T18:01:52.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=7672 rules.size()=3 2024-03-04T18:01:52.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=7672 2024-03-04T18:01:52.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=7672 stripe_ofs=7672 part_ofs=7672 rule->part_size=7671 2024-03-04T18:01:52.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=7671 rules.size()=3 2024-03-04T18:01:52.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4201976 part_ofs=7672 rule->part_size=7671 2024-03-04T18:01:52.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=15343 stripe_ofs=15343 part_ofs=15343 rule->part_size=1035 2024-03-04T18:01:52.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=1035 rules.size()=3 2024-03-04T18:01:52.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4209647 part_ofs=15343 rule->part_size=1035 2024-03-04T18:01:52.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=16378 stripe_ofs=16378 part_ofs=16378 rule->part_size=1035 2024-03-04T18:01:52.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.2_e455ae1c2e5a219f513d7aaddc21adc8 2024-03-04T18:01:52.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:52.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.1 2024-03-04T18:01:52.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 440 2024-03-04T18:01:52.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.2 2024-03-04T18:01:52.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 704 2024-03-04T18:01:52.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.3 2024-03-04T18:01:52.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 968 2024-03-04T18:01:52.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:52.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.984+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.2_e455ae1c2e5a219f513d7aaddc21adc8 2024-03-04T18:01:52.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.988+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2024-03-04T18:01:52.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.988+0000 7f3f691dc980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:01:52.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.988+0000 7f3f691dc980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-03-04T18:01:52.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.988+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:01:52.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.988+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:01:52.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.988+0000 7f3f691dc980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:01:52.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.988+0000 7f3f691dc980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:01:52.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.988+0000 7f3f691dc980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17])"; uploads.size()=0; is_truncated=0 2024-03-04T18:01:52.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.992+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2024-03-04T18:01:52.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.992+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:52.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.992+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=243 2024-03-04T18:01:52.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.992+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x11 2024-03-04T18:01:52.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.992+0000 7f3f691dc980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2024-03-04T18:01:52.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.996+0000 7f3f691dc980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2024-03-04T18:01:52.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.996+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1, ofs0, ns] 2024-03-04T18:01:52.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:52.996+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627786 cookie 94212396652064 notifier 5889 bl.length()=199 2024-03-04T18:01:53.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.004+0000 7f3f691dc980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17 from cache 2024-03-04T18:01:53.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.004+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17, ofs0, ns] 2024-03-04T18:01:53.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.004+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627797 cookie 94212396648080 notifier 5889 bl.length()=257 2024-03-04T18:01:53.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.008+0000 7f3f691dc980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17 : miss 2024-03-04T18:01:53.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.008+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.012+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:53.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.012+0000 7f3f691dc980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17 info.flags=0x0 2024-03-04T18:01:53.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.012+0000 7f3f691dc980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17 to cache LRU end 2024-03-04T18:01:53.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.016+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15 : miss 2024-03-04T18:01:53.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.016+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=412 2024-03-04T18:01:53.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15 info.flags=0x17 2024-03-04T18:01:53.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15 to cache LRU end 2024-03-04T18:01:53.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:01:53.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:01:53.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15 2024-03-04T18:01:53.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-03-04T18:01:53.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-03-04T18:01:53.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-03-04T18:01:53.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 20 get_obj_state: octx=0x55af886f6cd0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x55af88732258 s->prefetch_data=0 2024-03-04T18:01:53.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 10 manifest: total_size = 11534336 2024-03-04T18:01:53.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4585518418313471972 2024-03-04T18:01:53.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 20 get_obj_state: octx=0x55af886f6cd0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x55af88732258 s->prefetch_data=0 2024-03-04T18:01:53.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88732258 2024-03-04T18:01:53.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.020+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15.0 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-03-04T18:01:53.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4585518418313471972 2024-03-04T18:01:53.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:53.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-FROZEN.2~-8noU2j1nj9CEob3DnwSrTLOYWTYQGl.1 2024-03-04T18:01:53.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 367 2024-03-04T18:01:53.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__shadow_foo-FROZEN.2~-8noU2j1nj9CEob3DnwSrTLOYWTYQGl.1_1 2024-03-04T18:01:53.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 632 2024-03-04T18:01:53.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-FROZEN.2~-8noU2j1nj9CEob3DnwSrTLOYWTYQGl.2 2024-03-04T18:01:53.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 899 2024-03-04T18:01:53.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__shadow_foo-FROZEN.2~-8noU2j1nj9CEob3DnwSrTLOYWTYQGl.2_1 2024-03-04T18:01:53.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1164 2024-03-04T18:01:53.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-FROZEN.2~-8noU2j1nj9CEob3DnwSrTLOYWTYQGl.3 2024-03-04T18:01:53.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1431 2024-03-04T18:01:53.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:53.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.028+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.10tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4585518418313471972 2024-03-04T18:01:53.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.032+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-03-04T18:01:53.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.032+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f6000fff0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x55af8851c588 s->prefetch_data=0 2024-03-04T18:01:53.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.032+0000 7f3f691dc980 10 manifest: total_size = 11534336 2024-03-04T18:01:53.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.032+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2499779498388887256 2024-03-04T18:01:53.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.032+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f6000fff0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x55af8851c588 s->prefetch_data=0 2024-03-04T18:01:53.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.032+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8851c588 2024-03-04T18:01:53.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.032+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15.0 2024-03-04T18:01:53.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:53.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:53.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:53.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2499779498388887256 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-LUKEWARM.2~OYMnPgqQAanTFlMmjgFQjXzvKfR2zVn.1 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 373 2024-03-04T18:01:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__shadow_foo-LUKEWARM.2~OYMnPgqQAanTFlMmjgFQjXzvKfR2zVn.1_1 2024-03-04T18:01:53.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 644 2024-03-04T18:01:53.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-LUKEWARM.2~OYMnPgqQAanTFlMmjgFQjXzvKfR2zVn.2 2024-03-04T18:01:53.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 917 2024-03-04T18:01:53.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__shadow_foo-LUKEWARM.2~OYMnPgqQAanTFlMmjgFQjXzvKfR2zVn.2_1 2024-03-04T18:01:53.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1188 2024-03-04T18:01:53.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-LUKEWARM.2~OYMnPgqQAanTFlMmjgFQjXzvKfR2zVn.3 2024-03-04T18:01:53.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1461 2024-03-04T18:01:53.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:53.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.036+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2499779498388887256 2024-03-04T18:01:53.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.040+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-03-04T18:01:53.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.040+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60050570 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x55af8851c588 s->prefetch_data=0 2024-03-04T18:01:53.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.044+0000 7f3f691dc980 10 manifest: total_size = 16377 2024-03-04T18:01:53.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.044+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17231879429835817372 2024-03-04T18:01:53.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.044+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60050570 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x55af8851c588 s->prefetch_data=0 2024-03-04T18:01:53.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.044+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8851c588 2024-03-04T18:01:53.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.044+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15.0 2024-03-04T18:01:53.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=7671 rules.size()=2 2024-03-04T18:01:53.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=7671 2024-03-04T18:01:53.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=7671 stripe_ofs=7671 part_ofs=7671 rule->part_size=7671 2024-03-04T18:01:53.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=7671 rules.size()=2 2024-03-04T18:01:53.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4201975 part_ofs=7671 rule->part_size=7671 2024-03-04T18:01:53.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=15342 stripe_ofs=15342 part_ofs=15342 rule->part_size=1035 2024-03-04T18:01:53.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=1035 rules.size()=2 2024-03-04T18:01:53.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 RGWObjManifest::operator++(): stripe_ofs=4209646 part_ofs=15342 rule->part_size=1035 2024-03-04T18:01:53.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=16377 stripe_ofs=16377 part_ofs=16377 rule->part_size=1035 2024-03-04T18:01:53.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17231879429835817372 2024-03-04T18:01:53.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:53.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-STANDARD.2~J4gplH8e0-_OUb8Iirm60u9-a4HqabO.1 2024-03-04T18:01:53.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 365 2024-03-04T18:01:53.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-STANDARD.2~J4gplH8e0-_OUb8Iirm60u9-a4HqabO.2 2024-03-04T18:01:53.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 629 2024-03-04T18:01:53.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-STANDARD.2~J4gplH8e0-_OUb8Iirm60u9-a4HqabO.3 2024-03-04T18:01:53.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 893 2024-03-04T18:01:53.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:53.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.048+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17231879429835817372 2024-03-04T18:01:53.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.052+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-03-04T18:01:53.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.052+0000 7f3f691dc980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:01:53.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.052+0000 7f3f691dc980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-03-04T18:01:53.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.052+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:01:53.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.052+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:01:53.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.052+0000 7f3f691dc980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:01:53.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.052+0000 7f3f691dc980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:01:53.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.052+0000 7f3f691dc980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15])"; uploads.size()=0; is_truncated=0 2024-03-04T18:01:53.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.056+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2024-03-04T18:01:53.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.056+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.056+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=238 2024-03-04T18:01:53.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.056+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x11 2024-03-04T18:01:53.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.056+0000 7f3f691dc980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2024-03-04T18:01:53.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.060+0000 7f3f691dc980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2024-03-04T18:01:53.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.060+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1, ofs0, ns] 2024-03-04T18:01:53.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.060+0000 7f3e49ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627798 cookie 94212396648080 notifier 5889 bl.length()=194 2024-03-04T18:01:53.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.068+0000 7f3f691dc980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15 from cache 2024-03-04T18:01:53.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.068+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15, ofs0, ns] 2024-03-04T18:01:53.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.068+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627799 cookie 94212396648080 notifier 5889 bl.length()=252 2024-03-04T18:01:53.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.076+0000 7f3f691dc980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15 : miss 2024-03-04T18:01:53.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.076+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.076+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:53.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.076+0000 7f3f691dc980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15 info.flags=0x0 2024-03-04T18:01:53.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.076+0000 7f3f691dc980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15 to cache LRU end 2024-03-04T18:01:53.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.080+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14 : miss 2024-03-04T18:01:53.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.080+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=402 2024-03-04T18:01:53.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14 info.flags=0x17 2024-03-04T18:01:53.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14 to cache LRU end 2024-03-04T18:01:53.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:01:53.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:01:53.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14 2024-03-04T18:01:53.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-03-04T18:01:53.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-03-04T18:01:53.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-03-04T18:01:53.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8872d3e0 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x55af88746968 s->prefetch_data=0 2024-03-04T18:01:53.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 10 manifest: total_size = 10485760 2024-03-04T18:01:53.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10390100625577012679 2024-03-04T18:01:53.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 20 get_obj_state: octx=0x55af8872d3e0 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x55af88746968 s->prefetch_data=0 2024-03-04T18:01:53.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88746968 2024-03-04T18:01:53.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.084+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14.0 2024-03-04T18:01:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-03-04T18:01:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2024-03-04T18:01:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-03-04T18:01:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10390100625577012679 2024-03-04T18:01:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14__shadow_.4YDwkBT-ECBPSaihDhWwP99qJf6xg1o_0 2024-03-04T18:01:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 338 2024-03-04T18:01:53.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14__shadow_.4YDwkBT-ECBPSaihDhWwP99qJf6xg1o_1 2024-03-04T18:01:53.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 575 2024-03-04T18:01:53.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14__shadow_.4YDwkBT-ECBPSaihDhWwP99qJf6xg1o_2 2024-03-04T18:01:53.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 812 2024-03-04T18:01:53.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:53.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.092+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10390100625577012679 2024-03-04T18:01:53.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.096+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-03-04T18:01:53.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.096+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f58064a40 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x55af8851e1b8 s->prefetch_data=0 2024-03-04T18:01:53.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.096+0000 7f3f691dc980 10 manifest: total_size = 10485760 2024-03-04T18:01:53.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.096+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16122458151288315120 2024-03-04T18:01:53.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.096+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f58064a40 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x55af8851e1b8 s->prefetch_data=0 2024-03-04T18:01:53.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.096+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8851e1b8 2024-03-04T18:01:53.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.096+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14.0 2024-03-04T18:01:53.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:53.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-03-04T18:01:53.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:53.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2024-03-04T18:01:53.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-03-04T18:01:53.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-03-04T18:01:53.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16122458151288315120 2024-03-04T18:01:53.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-03-04T18:01:53.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14__shadow_.xRJ6_oVzZJ4Pc0ktCPGTWZFl7mDxVSU_0 2024-03-04T18:01:53.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 340 2024-03-04T18:01:53.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14__shadow_.xRJ6_oVzZJ4Pc0ktCPGTWZFl7mDxVSU_1 2024-03-04T18:01:53.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 579 2024-03-04T18:01:53.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14__shadow_.xRJ6_oVzZJ4Pc0ktCPGTWZFl7mDxVSU_2 2024-03-04T18:01:53.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 818 2024-03-04T18:01:53.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-03-04T18:01:53.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.104+0000 7f3f691dc980 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16122458151288315120 2024-03-04T18:01:53.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.108+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-03-04T18:01:53.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.108+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f58064a40 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x55af8851e1b8 s->prefetch_data=0 2024-03-04T18:01:53.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.108+0000 7f3f691dc980 10 manifest: total_size = 649292 2024-03-04T18:01:53.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.108+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11237400000285690591 2024-03-04T18:01:53.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.108+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f58064a40 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x55af8851e1b8 s->prefetch_data=0 2024-03-04T18:01:53.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.108+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8851e1b8 2024-03-04T18:01:53.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.108+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14.0 2024-03-04T18:01:53.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.112+0000 7f3f691dc980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:01:53.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.112+0000 7f3f691dc980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-03-04T18:01:53.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.116+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-03-04T18:01:53.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.116+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:01:53.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.116+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:01:53.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.116+0000 7f3f691dc980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:01:53.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.116+0000 7f3f691dc980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:01:53.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.116+0000 7f3f691dc980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14])"; uploads.size()=0; is_truncated=0 2024-03-04T18:01:53.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.120+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2024-03-04T18:01:53.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.120+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.120+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=228 2024-03-04T18:01:53.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.120+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x11 2024-03-04T18:01:53.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.120+0000 7f3f691dc980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2024-03-04T18:01:53.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.124+0000 7f3f691dc980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2024-03-04T18:01:53.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.124+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1, ofs0, ns] 2024-03-04T18:01:53.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.124+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627807 cookie 94212396651008 notifier 5889 bl.length()=184 2024-03-04T18:01:53.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.132+0000 7f3f691dc980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14 from cache 2024-03-04T18:01:53.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.132+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14, ofs0, ns] 2024-03-04T18:01:53.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.132+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627856 cookie 94212396655056 notifier 5889 bl.length()=242 2024-03-04T18:01:53.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.136+0000 7f3f691dc980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14 : miss 2024-03-04T18:01:53.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.136+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.140+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:53.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.140+0000 7f3f691dc980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14 info.flags=0x0 2024-03-04T18:01:53.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.140+0000 7f3f691dc980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14 to cache LRU end 2024-03-04T18:01:53.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.144+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7 : miss 2024-03-04T18:01:53.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.144+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=397 2024-03-04T18:01:53.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7 info.flags=0x17 2024-03-04T18:01:53.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7 to cache LRU end 2024-03-04T18:01:53.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:01:53.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:01:53.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7 2024-03-04T18:01:53.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 10 cls_bucket_list_unordered: got __ 2024-03-04T18:01:53.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 10 cls_bucket_list_unordered: got __ _ 2024-03-04T18:01:53.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 10 cls_bucket_list_unordered: got ___ 2024-03-04T18:01:53.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 10 cls_bucket_list_unordered: got obj 2024-03-04T18:01:53.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88728bf0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 10 manifest: total_size = 0 2024-03-04T18:01:53.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8575713010236149961 2024-03-04T18:01:53.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88728bf0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8851c3a8 2024-03-04T18:01:53.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.148+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7.0 2024-03-04T18:01:53.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.156+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88728bf0 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.156+0000 7f3f691dc980 10 manifest: total_size = 0 2024-03-04T18:01:53.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.156+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17818385103965946844 2024-03-04T18:01:53.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.156+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88728bf0 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.156+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8851c3a8 2024-03-04T18:01:53.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.156+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7.0 2024-03-04T18:01:53.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.156+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=__ 2024-03-04T18:01:53.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.160+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88728bf0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.164+0000 7f3f691dc980 10 manifest: total_size = 0 2024-03-04T18:01:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.164+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7043790851849200377 2024-03-04T18:01:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.164+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88728bf0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.164+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8851c3a8 2024-03-04T18:01:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.164+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7.0 2024-03-04T18:01:53.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.164+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=__ _ 2024-03-04T18:01:53.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.168+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60001eb0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.168+0000 7f3f691dc980 10 manifest: total_size = 0 2024-03-04T18:01:53.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.168+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11360167880985233985 2024-03-04T18:01:53.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.168+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60001eb0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.168+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8851c3a8 2024-03-04T18:01:53.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.168+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7.0 2024-03-04T18:01:53.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.172+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=___ 2024-03-04T18:01:53.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.176+0000 7f3f691dc980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:01:53.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.176+0000 7f3f691dc980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-03-04T18:01:53.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.176+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=obj 2024-03-04T18:01:53.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.180+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:01:53.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.180+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:01:53.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.180+0000 7f3f691dc980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:01:53.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.180+0000 7f3f691dc980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:01:53.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.180+0000 7f3f691dc980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7])"; uploads.size()=0; is_truncated=0 2024-03-04T18:01:53.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.184+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2024-03-04T18:01:53.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.184+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.184+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=223 2024-03-04T18:01:53.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.184+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x11 2024-03-04T18:01:53.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.184+0000 7f3f691dc980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2024-03-04T18:01:53.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.188+0000 7f3f691dc980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2024-03-04T18:01:53.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.188+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.1, ofs0, ns] 2024-03-04T18:01:53.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.188+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627857 cookie 94212396646864 notifier 5889 bl.length()=181 2024-03-04T18:01:53.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.192+0000 7f3f691dc980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7 from cache 2024-03-04T18:01:53.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.192+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7, ofs0, ns] 2024-03-04T18:01:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.192+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627808 cookie 94212396651008 notifier 5889 bl.length()=238 2024-03-04T18:01:53.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.200+0000 7f3f691dc980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7 : miss 2024-03-04T18:01:53.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.200+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.200+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:53.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.200+0000 7f3f691dc980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7 info.flags=0x0 2024-03-04T18:01:53.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.200+0000 7f3f691dc980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7 to cache LRU end 2024-03-04T18:01:53.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8 : miss 2024-03-04T18:01:53.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=397 2024-03-04T18:01:53.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8 info.flags=0x17 2024-03-04T18:01:53.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8 to cache LRU end 2024-03-04T18:01:53.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:01:53.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:01:53.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8 2024-03-04T18:01:53.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 10 cls_bucket_list_unordered: got __ 2024-03-04T18:01:53.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 10 cls_bucket_list_unordered: got __ _ 2024-03-04T18:01:53.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 10 cls_bucket_list_unordered: got ___ 2024-03-04T18:01:53.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 10 cls_bucket_list_unordered: got obj 2024-03-04T18:01:53.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60050570 obj=test-client.0-r-test-small-obj-data.2:__ state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 10 manifest: total_size = 524288 2024-03-04T18:01:53.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.965125972830607066 2024-03-04T18:01:53.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60050570 obj=test-client.0-r-test-small-obj-data.2:__ state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8851c3a8 2024-03-04T18:01:53.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.208+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8.0 2024-03-04T18:01:53.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.216+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60050570 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.216+0000 7f3f691dc980 10 manifest: total_size = 524288 2024-03-04T18:01:53.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.216+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.257653359125947608 2024-03-04T18:01:53.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.216+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60050570 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.216+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8851c3a8 2024-03-04T18:01:53.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.216+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8.0 2024-03-04T18:01:53.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.220+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=__ 2024-03-04T18:01:53.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.224+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60050570 obj=test-client.0-r-test-small-obj-data.2:___ state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.224+0000 7f3f691dc980 10 manifest: total_size = 2078 2024-03-04T18:01:53.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.224+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.3115968915927407062 2024-03-04T18:01:53.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.224+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60050570 obj=test-client.0-r-test-small-obj-data.2:___ state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.224+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8851c3a8 2024-03-04T18:01:53.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.224+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8.0 2024-03-04T18:01:53.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.224+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=__ _ 2024-03-04T18:01:53.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.228+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60050570 obj=test-client.0-r-test-small-obj-data.2:obj state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.228+0000 7f3f691dc980 10 manifest: total_size = 24595 2024-03-04T18:01:53.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.228+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14060043296062688245 2024-03-04T18:01:53.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.228+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60050570 obj=test-client.0-r-test-small-obj-data.2:obj state=0x55af8851c3a8 s->prefetch_data=0 2024-03-04T18:01:53.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.228+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af8851c3a8 2024-03-04T18:01:53.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.228+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8.0 2024-03-04T18:01:53.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.232+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=___ 2024-03-04T18:01:53.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.236+0000 7f3f691dc980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:01:53.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.236+0000 7f3f691dc980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-03-04T18:01:53.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.236+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=obj 2024-03-04T18:01:53.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.240+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:01:53.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.240+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:01:53.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.240+0000 7f3f691dc980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:01:53.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.240+0000 7f3f691dc980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:01:53.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.240+0000 7f3f691dc980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8])"; uploads.size()=0; is_truncated=0 2024-03-04T18:01:53.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.244+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2024-03-04T18:01:53.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.244+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.244+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=223 2024-03-04T18:01:53.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.244+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x11 2024-03-04T18:01:53.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.244+0000 7f3f691dc980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2024-03-04T18:01:53.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.248+0000 7f3f691dc980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2024-03-04T18:01:53.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.248+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.2, ofs0, ns] 2024-03-04T18:01:53.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.248+0000 7f3e49ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627858 cookie 94212396646864 notifier 5889 bl.length()=181 2024-03-04T18:01:53.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.252+0000 7f3f691dc980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8 from cache 2024-03-04T18:01:53.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.252+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8, ofs0, ns] 2024-03-04T18:01:53.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.256+0000 7f3e49ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627809 cookie 94212396651008 notifier 5889 bl.length()=238 2024-03-04T18:01:53.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.260+0000 7f3f691dc980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8 : miss 2024-03-04T18:01:53.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.260+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.260+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:53.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.260+0000 7f3f691dc980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8 info.flags=0x0 2024-03-04T18:01:53.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.260+0000 7f3f691dc980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8 to cache LRU end 2024-03-04T18:01:53.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.268+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9 : miss 2024-03-04T18:01:53.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.268+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.268+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=397 2024-03-04T18:01:53.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.268+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9 info.flags=0x17 2024-03-04T18:01:53.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.268+0000 7f3f691dc980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9 to cache LRU end 2024-03-04T18:01:53.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.268+0000 7f3f691dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:01:53.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.268+0000 7f3f691dc980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-03-04T18:01:53.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.268+0000 7f3f691dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9 2024-03-04T18:01:53.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.272+0000 7f3f691dc980 10 cls_bucket_list_unordered: got __ 2024-03-04T18:01:53.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.272+0000 7f3f691dc980 10 cls_bucket_list_unordered: got __ _ 2024-03-04T18:01:53.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.272+0000 7f3f691dc980 10 cls_bucket_list_unordered: got ___ 2024-03-04T18:01:53.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.272+0000 7f3f691dc980 10 cls_bucket_list_unordered: got obj 2024-03-04T18:01:53.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.272+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88734720 obj=test-client.0-r-test-small-obj-data.3:__ state=0x55af88746968 s->prefetch_data=0 2024-03-04T18:01:53.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.272+0000 7f3f691dc980 10 manifest: total_size = 49187 2024-03-04T18:01:53.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.272+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11117538830169498865 2024-03-04T18:01:53.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.272+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88734720 obj=test-client.0-r-test-small-obj-data.3:__ state=0x55af88746968 s->prefetch_data=0 2024-03-04T18:01:53.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.272+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88746968 2024-03-04T18:01:53.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.272+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9.0 2024-03-04T18:01:53.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.280+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88734720 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x55af88746968 s->prefetch_data=0 2024-03-04T18:01:53.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.280+0000 7f3f691dc980 10 manifest: total_size = 1035 2024-03-04T18:01:53.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.280+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10729385446242513105 2024-03-04T18:01:53.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.280+0000 7f3f691dc980 20 get_obj_state: octx=0x55af88734720 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x55af88746968 s->prefetch_data=0 2024-03-04T18:01:53.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.280+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88746968 2024-03-04T18:01:53.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.280+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9.0 2024-03-04T18:01:53.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.280+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=__ 2024-03-04T18:01:53.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.288+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f6004ff80 obj=test-client.0-r-test-small-obj-data.3:___ state=0x55af88746968 s->prefetch_data=0 2024-03-04T18:01:53.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.288+0000 7f3f691dc980 10 manifest: total_size = 55 2024-03-04T18:01:53.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.288+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.231880375105239533 2024-03-04T18:01:53.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.288+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f6004ff80 obj=test-client.0-r-test-small-obj-data.3:___ state=0x55af88746968 s->prefetch_data=0 2024-03-04T18:01:53.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.288+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88746968 2024-03-04T18:01:53.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.288+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9.0 2024-03-04T18:01:53.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.292+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=__ _ 2024-03-04T18:01:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.292+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60067620 obj=test-client.0-r-test-small-obj-data.3:obj state=0x55af88746968 s->prefetch_data=0 2024-03-04T18:01:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.296+0000 7f3f691dc980 10 manifest: total_size = 4134 2024-03-04T18:01:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.296+0000 7f3f691dc980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4322808485900590366 2024-03-04T18:01:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.296+0000 7f3f691dc980 20 get_obj_state: octx=0x7f3f60067620 obj=test-client.0-r-test-small-obj-data.3:obj state=0x55af88746968 s->prefetch_data=0 2024-03-04T18:01:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.296+0000 7f3f691dc980 20 prepare_atomic_modification: state is not atomic. state=0x55af88746968 2024-03-04T18:01:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.296+0000 7f3f691dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9.0 2024-03-04T18:01:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.296+0000 7f3e4a7fc640 20 handle_completion(): completion ok for obj=___ 2024-03-04T18:01:53.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.300+0000 7f3f691dc980 20 list_objects_ordered: starting attempt 1 2024-03-04T18:01:53.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.300+0000 7f3f691dc980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-03-04T18:01:53.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.304+0000 7f3e49ffb640 20 handle_completion(): completion ok for obj=obj 2024-03-04T18:01:53.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.304+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-04T18:01:53.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.304+0000 7f3f691dc980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-04T18:01:53.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.304+0000 7f3f691dc980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-04T18:01:53.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.304+0000 7f3f691dc980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-04T18:01:53.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.304+0000 7f3f691dc980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9])"; uploads.size()=0; is_truncated=0 2024-03-04T18:01:53.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.308+0000 7f3f691dc980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2024-03-04T18:01:53.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.308+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.308+0000 7f3f691dc980 20 rados_obj.operate() r=0 bl.length=223 2024-03-04T18:01:53.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.308+0000 7f3f691dc980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x11 2024-03-04T18:01:53.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.308+0000 7f3f691dc980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2024-03-04T18:01:53.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.312+0000 7f3f691dc980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2024-03-04T18:01:53.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.312+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.3, ofs0, ns] 2024-03-04T18:01:53.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.312+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627859 cookie 94212396646864 notifier 5889 bl.length()=181 2024-03-04T18:01:53.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.320+0000 7f3f691dc980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9 from cache 2024-03-04T18:01:53.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.320+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9, ofs0, ns] 2024-03-04T18:01:53.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.320+0000 7f3e49ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627810 cookie 94212396651008 notifier 5889 bl.length()=238 2024-03-04T18:01:53.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.328+0000 7f3f691dc980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9 : miss 2024-03-04T18:01:53.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.328+0000 7f3f691dc980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.328+0000 7f3f691dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:53.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.328+0000 7f3f691dc980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9 info.flags=0x0 2024-03-04T18:01:53.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.328+0000 7f3f691dc980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9 to cache LRU end 2024-03-04T18:01:53.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.332+0000 7f3f691dc980 10 removing key index: QOXRJIHOSDFWAPOBUJVJ 2024-03-04T18:01:53.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.336+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.keys:QOXRJIHOSDFWAPOBUJVJ, ofs0, ns] 2024-03-04T18:01:53.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.336+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627787 cookie 94212396652064 notifier 5889 bl.length()=170 2024-03-04T18:01:53.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.340+0000 7f3f691dc980 10 removing email index: ragweed.client.0+test@test.test 2024-03-04T18:01:53.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.344+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2024-03-04T18:01:53.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.344+0000 7f3e49ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627811 cookie 94212396651008 notifier 5889 bl.length()=182 2024-03-04T18:01:53.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.348+0000 7f3f691dc980 10 removing user buckets index 2024-03-04T18:01:53.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.352+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0.buckets, ofs0, ns] 2024-03-04T18:01:53.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.352+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627800 cookie 94212396648080 notifier 5889 bl.length()=173 2024-03-04T18:01:53.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.356+0000 7f3f691dc980 10 removing user index: ragweed.client.0 2024-03-04T18:01:53.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.356+0000 7f3f691dc980 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2024-03-04T18:01:53.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.356+0000 7f3f691dc980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2024-03-04T18:01:53.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.360+0000 7f3e4a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627860 cookie 94212396655056 notifier 5889 bl.length()=165 2024-03-04T18:01:53.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.368+0000 7f3e1d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:01:53.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.372+0000 7f3f691dc980 20 remove_watcher() i=0 2024-03-04T18:01:53.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.372+0000 7f3f691dc980 2 removed watcher, disabling cache 2024-03-04T18:01:53.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.376+0000 7f3f691dc980 20 remove_watcher() i=1 2024-03-04T18:01:53.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.380+0000 7f3f691dc980 20 remove_watcher() i=2 2024-03-04T18:01:53.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.380+0000 7f3f691dc980 20 remove_watcher() i=3 2024-03-04T18:01:53.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.384+0000 7f3f691dc980 20 remove_watcher() i=4 2024-03-04T18:01:53.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.384+0000 7f3f691dc980 20 remove_watcher() i=5 2024-03-04T18:01:53.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.388+0000 7f3f691dc980 20 remove_watcher() i=6 2024-03-04T18:01:53.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.392+0000 7f3f691dc980 20 remove_watcher() i=7 2024-03-04T18:01:53.410 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2024-03-04T18:01:53.487 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:01:53.488 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:01:53.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.524+0000 7effa9c68980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.528+0000 7effa9c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:53.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.528+0000 7effa9c68980 20 realm 2024-03-04T18:01:53.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.528+0000 7effa9c68980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.528+0000 7effa9c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:53.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.528+0000 7effa9c68980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:01:53.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.528+0000 7effa9c68980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.528+0000 7effa9c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:53.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.528+0000 7effa9c68980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.528+0000 7effa9c68980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:01:53.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.528+0000 7effa9c68980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.532+0000 7effa9c68980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:01:53.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.532+0000 7effa9c68980 20 searching for the correct realm 2024-03-04T18:01:53.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.536+0000 7effa7f81640 0 -- 172.21.15.137:0/751251429 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7effa00398a0 msgr2=0x7effa002a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:01:53.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.544+0000 7effa7f81640 0 -- 172.21.15.137:0/751251429 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x56425337daf0 msgr2=0x56425337ffb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:01:53.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.548+0000 7effa9c68980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:01:53.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.548+0000 7effa9c68980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:01:53.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.548+0000 7effa9c68980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:01:53.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.548+0000 7effa9c68980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:01:53.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.548+0000 7effa9c68980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:01:53.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.548+0000 7effa9c68980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:01:53.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.548+0000 7effa9c68980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.548+0000 7effa9c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:53.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.548+0000 7effa9c68980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:01:53.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:01:53.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 20 zone default found 2024-03-04T18:01:53.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 4 Realm: () 2024-03-04T18:01:53.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:01:53.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:01:53.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:01:53.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 20 zonegroup default 2024-03-04T18:01:53.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:53.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:53.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:01:53.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 20 started sync module instance, tier type = 2024-03-04T18:01:53.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.552+0000 7effa9c68980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:01:53.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.568+0000 7effa9c68980 20 add_watcher() i=0 2024-03-04T18:01:53.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.568+0000 7effa9c68980 20 add_watcher() i=1 2024-03-04T18:01:53.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.568+0000 7effa9c68980 20 add_watcher() i=2 2024-03-04T18:01:53.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.568+0000 7effa9c68980 20 add_watcher() i=3 2024-03-04T18:01:53.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.568+0000 7effa9c68980 20 add_watcher() i=4 2024-03-04T18:01:53.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.568+0000 7effa9c68980 20 add_watcher() i=5 2024-03-04T18:01:53.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.568+0000 7effa9c68980 20 add_watcher() i=6 2024-03-04T18:01:53.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.572+0000 7effa9c68980 20 add_watcher() i=7 2024-03-04T18:01:53.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.572+0000 7effa9c68980 2 all 8 watchers are set, enabling cache 2024-03-04T18:01:53.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.576+0000 7efe5dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:01:53.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.576+0000 7effa9c68980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:01:53.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.576+0000 7effa9c68980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:01:53.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.652+0000 7efe5cfe1640 20 reqs_thread_entry: start 2024-03-04T18:01:53.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.652+0000 7effa9c68980 20 init_complete bucket index max shards: 1 2024-03-04T18:01:53.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.652+0000 7effa9c68980 20 Filter name: none 2024-03-04T18:01:53.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.652+0000 7efdd67fc640 20 reqs_thread_entry: start 2024-03-04T18:01:53.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.652+0000 7effa9c68980 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2024-03-04T18:01:53.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.652+0000 7effa9c68980 20 rados->read ofs=0 len=0 2024-03-04T18:01:53.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.652+0000 7effa9c68980 20 rados_obj.operate() r=0 bl.length=453 2024-03-04T18:01:53.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.652+0000 7effa9c68980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x13 2024-03-04T18:01:53.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.652+0000 7effa9c68980 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-03-04T18:01:53.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.652+0000 7effa9c68980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:01:53.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.652+0000 7effa9c68980 10 removing key index: BACTGXKRQHOLLBJINBBI 2024-03-04T18:01:53.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.656+0000 7effa9c68980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.keys:BACTGXKRQHOLLBJINBBI, ofs0, ns] 2024-03-04T18:01:53.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.656+0000 7efe8affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627801 cookie 94842864024640 notifier 5904 bl.length()=170 2024-03-04T18:01:53.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.656+0000 7effa9c68980 10 removing email index: sysuser.client.0+test@test.test 2024-03-04T18:01:53.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.660+0000 7effa9c68980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2024-03-04T18:01:53.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.660+0000 7efe8affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627861 cookie 94842864021088 notifier 5904 bl.length()=182 2024-03-04T18:01:53.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.660+0000 7effa9c68980 10 removing user buckets index 2024-03-04T18:01:53.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.664+0000 7effa9c68980 10 removing user index: sysuser.client.0 2024-03-04T18:01:53.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.668+0000 7effa9c68980 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2024-03-04T18:01:53.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.668+0000 7effa9c68980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2024-03-04T18:01:53.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.668+0000 7efe8a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627812 cookie 94842864016160 notifier 5904 bl.length()=165 2024-03-04T18:01:53.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.668+0000 7efe5dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:01:53.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.672+0000 7effa9c68980 20 remove_watcher() i=0 2024-03-04T18:01:53.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.672+0000 7effa9c68980 2 removed watcher, disabling cache 2024-03-04T18:01:53.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.676+0000 7effa9c68980 20 remove_watcher() i=1 2024-03-04T18:01:53.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.676+0000 7effa9c68980 20 remove_watcher() i=2 2024-03-04T18:01:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.680+0000 7effa9c68980 20 remove_watcher() i=3 2024-03-04T18:01:53.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.680+0000 7effa9c68980 20 remove_watcher() i=4 2024-03-04T18:01:53.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.684+0000 7effa9c68980 20 remove_watcher() i=5 2024-03-04T18:01:53.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.688+0000 7effa9c68980 20 remove_watcher() i=6 2024-03-04T18:01:53.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:01:53.688+0000 7effa9c68980 20 remove_watcher() i=7 2024-03-04T18:01:53.709 INFO:tasks.ragweed:Removing ragweed... 2024-03-04T18:01:53.709 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/ragweed.client.0 2024-03-04T18:01:53.802 INFO:teuthology.run_tasks:Running task workunit... 2024-03-04T18:01:53.812 INFO:tasks.workunit:Pulling workunits from ref 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T18:01:53.812 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-04T18:01:53.812 DEBUG:teuthology.orchestra.run.smithi137:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-04T18:01:53.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T18:01:53.819 INFO:teuthology.orchestra.run.smithi137.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-04T18:01:53.819 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T18:01:53.865 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-04T18:01:53.865 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-04T18:01:53.916 INFO:tasks.workunit:timeout=3h 2024-03-04T18:01:53.916 INFO:tasks.workunit:cleanup=True 2024-03-04T18:01:53.917 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/pritha-srivastava/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3aec390336fdfdc8f927818b718cfb0361bd47c8 2024-03-04T18:01:53.966 INFO:tasks.workunit.client.0.smithi137.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-04T18:02:52.974 INFO:tasks.workunit.client.0.smithi137.stderr:Note: switching to '3aec390336fdfdc8f927818b718cfb0361bd47c8'. 2024-03-04T18:02:52.974 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:02:52.974 INFO:tasks.workunit.client.0.smithi137.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T18:02:52.974 INFO:tasks.workunit.client.0.smithi137.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T18:02:52.974 INFO:tasks.workunit.client.0.smithi137.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T18:02:52.975 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:02:52.975 INFO:tasks.workunit.client.0.smithi137.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T18:02:52.975 INFO:tasks.workunit.client.0.smithi137.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T18:02:52.975 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:02:52.975 INFO:tasks.workunit.client.0.smithi137.stderr: git switch -c 2024-03-04T18:02:52.975 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:02:52.975 INFO:tasks.workunit.client.0.smithi137.stderr:Or undo this operation with: 2024-03-04T18:02:52.975 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:02:52.975 INFO:tasks.workunit.client.0.smithi137.stderr: git switch - 2024-03-04T18:02:52.975 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:02:52.975 INFO:tasks.workunit.client.0.smithi137.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T18:02:52.976 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:02:52.976 INFO:tasks.workunit.client.0.smithi137.stderr:HEAD is now at 3aec390336f qa/d4n: Start redis up before rgw 2024-03-04T18:02:52.981 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-04T18:02:53.034 INFO:tasks.workunit.client.0.smithi137.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T18:02:53.037 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T18:02:53.037 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T18:02:53.095 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T18:02:53.148 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T18:02:53.196 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T18:02:53.198 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T18:02:53.198 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T18:02:53.246 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T18:02:53.250 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T18:02:53.250 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-04T18:02:53.297 INFO:tasks.workunit:Running workunits matching rgw/run-reshard.sh on client.0... 2024-03-04T18:02:53.298 INFO:tasks.workunit:Running workunit rgw/run-reshard.sh... 2024-03-04T18:02:53.299 DEBUG:teuthology.orchestra.run.smithi137:workunit test rgw/run-reshard.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3aec390336fdfdc8f927818b718cfb0361bd47c8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2024-03-04T18:02:53.351 INFO:tasks.workunit.client.0.smithi137.stderr:+ ulimit -c 0 2024-03-04T18:02:53.351 INFO:tasks.workunit.client.0.smithi137.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2024-03-04T18:02:53.352 INFO:tasks.workunit.client.0.smithi137.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T18:02:53.352 INFO:tasks.workunit.client.0.smithi137.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T18:02:56.251 INFO:tasks.workunit.client.0.smithi137.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-03-04T18:02:56.251 INFO:tasks.workunit.client.0.smithi137.stderr:++ deactivate nondestructive 2024-03-04T18:02:56.251 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n '' ']' 2024-03-04T18:02:56.252 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n '' ']' 2024-03-04T18:02:56.252 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T18:02:56.252 INFO:tasks.workunit.client.0.smithi137.stderr:++ hash -r 2024-03-04T18:02:56.252 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n '' ']' 2024-03-04T18:02:56.252 INFO:tasks.workunit.client.0.smithi137.stderr:++ unset VIRTUAL_ENV 2024-03-04T18:02:56.252 INFO:tasks.workunit.client.0.smithi137.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-03-04T18:02:56.252 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-03-04T18:02:56.252 INFO:tasks.workunit.client.0.smithi137.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-03-04T18:02:56.252 INFO:tasks.workunit.client.0.smithi137.stderr:++ export VIRTUAL_ENV 2024-03-04T18:02:56.252 INFO:tasks.workunit.client.0.smithi137.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T18:02:56.252 INFO:tasks.workunit.client.0.smithi137.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T18:02:56.253 INFO:tasks.workunit.client.0.smithi137.stderr:++ export PATH 2024-03-04T18:02:56.253 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n '' ']' 2024-03-04T18:02:56.253 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -z '' ']' 2024-03-04T18:02:56.253 INFO:tasks.workunit.client.0.smithi137.stderr:++ _OLD_VIRTUAL_PS1= 2024-03-04T18:02:56.253 INFO:tasks.workunit.client.0.smithi137.stderr:++ PS1='(rgw) ' 2024-03-04T18:02:56.253 INFO:tasks.workunit.client.0.smithi137.stderr:++ export PS1 2024-03-04T18:02:56.253 INFO:tasks.workunit.client.0.smithi137.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-03-04T18:02:56.253 INFO:tasks.workunit.client.0.smithi137.stderr:++ export VIRTUAL_ENV_PROMPT 2024-03-04T18:02:56.253 INFO:tasks.workunit.client.0.smithi137.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T18:02:56.253 INFO:tasks.workunit.client.0.smithi137.stderr:++ hash -r 2024-03-04T18:02:56.253 INFO:tasks.workunit.client.0.smithi137.stderr:+ pip install pip --upgrade 2024-03-04T18:02:56.621 INFO:tasks.workunit.client.0.smithi137.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-03-04T18:02:56.864 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting pip 2024-03-04T18:02:56.871 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-03-04T18:02:56.907 INFO:tasks.workunit.client.0.smithi137.stdout:Installing collected packages: pip 2024-03-04T18:02:56.908 INFO:tasks.workunit.client.0.smithi137.stdout: Attempting uninstall: pip 2024-03-04T18:02:56.908 INFO:tasks.workunit.client.0.smithi137.stdout: Found existing installation: pip 22.0.2 2024-03-04T18:02:57.166 INFO:tasks.workunit.client.0.smithi137.stdout: Uninstalling pip-22.0.2: 2024-03-04T18:02:57.172 INFO:tasks.workunit.client.0.smithi137.stdout: Successfully uninstalled pip-22.0.2 2024-03-04T18:02:58.183 INFO:tasks.workunit.client.0.smithi137.stdout:Successfully installed pip-24.0 2024-03-04T18:02:58.252 INFO:tasks.workunit.client.0.smithi137.stderr:+ pip install boto3 2024-03-04T18:02:59.152 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting boto3 2024-03-04T18:02:59.154 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached boto3-1.34.54-py3-none-any.whl.metadata (6.6 kB) 2024-03-04T18:02:59.696 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting botocore<1.35.0,>=1.34.54 (from boto3) 2024-03-04T18:02:59.697 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached botocore-1.34.54-py3-none-any.whl.metadata (5.7 kB) 2024-03-04T18:02:59.759 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-03-04T18:02:59.761 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-03-04T18:02:59.827 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-03-04T18:02:59.829 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached s3transfer-0.10.0-py3-none-any.whl.metadata (1.7 kB) 2024-03-04T18:02:59.907 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T18:02:59.909 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-03-04T18:03:00.017 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting urllib3<2.1,>=1.25.4 (from botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T18:03:00.018 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB) 2024-03-04T18:03:00.238 INFO:tasks.workunit.client.0.smithi137.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.54->boto3) 2024-03-04T18:03:00.239 INFO:tasks.workunit.client.0.smithi137.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-03-04T18:03:00.253 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached boto3-1.34.54-py3-none-any.whl (139 kB) 2024-03-04T18:03:00.255 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached botocore-1.34.54-py3-none-any.whl (12.0 MB) 2024-03-04T18:03:00.279 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-03-04T18:03:00.280 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached s3transfer-0.10.0-py3-none-any.whl (82 kB) 2024-03-04T18:03:00.282 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-03-04T18:03:00.283 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached urllib3-2.0.7-py3-none-any.whl (124 kB) 2024-03-04T18:03:00.285 INFO:tasks.workunit.client.0.smithi137.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-03-04T18:03:00.384 INFO:tasks.workunit.client.0.smithi137.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-03-04T18:03:01.463 INFO:tasks.workunit.client.0.smithi137.stdout:Successfully installed boto3-1.34.54 botocore-1.34.54 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.0 six-1.16.0 urllib3-2.0.7 2024-03-04T18:03:01.618 INFO:tasks.workunit.client.0.smithi137.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_reshard.py 2024-03-04T18:03:01.756 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin user create --uid tester --display-name "Testing" --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn 2024-03-04T18:03:01.793 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:01.793 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:01.835 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:01.834+0000 7f9f35cf7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:01.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:01.838+0000 7f9f35cf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:01.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:01.838+0000 7f9f35cf7980 20 realm 2024-03-04T18:03:01.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:01.838+0000 7f9f35cf7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:01.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:01.838+0000 7f9f35cf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:01.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:01.838+0000 7f9f35cf7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:01.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:01.838+0000 7f9f35cf7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:01.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:01.838+0000 7f9f35cf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:01.838 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:01.838+0000 7f9f35cf7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:01.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:01.838+0000 7f9f35cf7980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:01.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:01.838+0000 7f9f35cf7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:01.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:01.838+0000 7f9f35cf7980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:01.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:01.838+0000 7f9f35cf7980 20 searching for the correct realm 2024-03-04T18:03:01.845 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:01.846+0000 7f9f34010640 0 -- 172.21.15.137:0/111213144 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f9f2c032400 msgr2=0x7f9f2c032880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:02.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.062+0000 7f9f34010640 0 -- 172.21.15.137:0/111213144 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x557e1cf65550 msgr2=0x557e1cf67a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:02.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.070+0000 7f9f35cf7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:02.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.070+0000 7f9f35cf7980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:02.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.070+0000 7f9f35cf7980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:02.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.070+0000 7f9f35cf7980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:02.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.070+0000 7f9f35cf7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:02.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.070+0000 7f9f35cf7980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:02.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.070+0000 7f9f35cf7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:02.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.070+0000 7f9f35cf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:02.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.070+0000 7f9f35cf7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:02.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.070+0000 7f9f35cf7980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:02.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.070+0000 7f9f35cf7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:02.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:02.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 20 zone default found 2024-03-04T18:03:02.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 4 Realm: () 2024-03-04T18:03:02.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:02.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:02.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:02.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 20 zonegroup default 2024-03-04T18:03:02.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:02.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:02.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:02.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:02.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:02.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:02.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 20 started sync module instance, tier type = 2024-03-04T18:03:02.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.074+0000 7f9f35cf7980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:02.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.098+0000 7f9f35cf7980 20 add_watcher() i=0 2024-03-04T18:03:02.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.098+0000 7f9f35cf7980 20 add_watcher() i=1 2024-03-04T18:03:02.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.098+0000 7f9f35cf7980 20 add_watcher() i=2 2024-03-04T18:03:02.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.098+0000 7f9f35cf7980 20 add_watcher() i=3 2024-03-04T18:03:02.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.098+0000 7f9f35cf7980 20 add_watcher() i=4 2024-03-04T18:03:02.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.098+0000 7f9f35cf7980 20 add_watcher() i=5 2024-03-04T18:03:02.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.098+0000 7f9f35cf7980 20 add_watcher() i=6 2024-03-04T18:03:02.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.102+0000 7f9f35cf7980 20 add_watcher() i=7 2024-03-04T18:03:02.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.102+0000 7f9f35cf7980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:02.104 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9de9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:02.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9f35cf7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:02.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9f35cf7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:02.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9f35cf7980 5 note: GC not initialized 2024-03-04T18:03:02.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9de8fe1640 20 reqs_thread_entry: start 2024-03-04T18:03:02.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9f35cf7980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:02.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9f35cf7980 20 Filter name: none 2024-03-04T18:03:02.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9f35cf7980 10 cache get: name=default.rgw.meta+users.uid+tester : miss 2024-03-04T18:03:02.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9f35cf7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:02.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9d627fc640 20 reqs_thread_entry: start 2024-03-04T18:03:02.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9f35cf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:02.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9f35cf7980 10 cache put: name=default.rgw.meta+users.uid+tester info.flags=0x0 2024-03-04T18:03:02.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9f35cf7980 10 adding default.rgw.meta+users.uid+tester to cache LRU end 2024-03-04T18:03:02.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9f35cf7980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-03-04T18:03:02.107 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.106+0000 7f9f35cf7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:02.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.110+0000 7f9f35cf7980 20 rados_obj.operate() r=0 bl.length=22 2024-03-04T18:03:02.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.110+0000 7f9f35cf7980 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x5 2024-03-04T18:03:02.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.110+0000 7f9f35cf7980 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-03-04T18:03:02.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.110+0000 7f9f35cf7980 10 cache get: name=default.rgw.meta+users.uid+rgw_datacache_user : miss 2024-03-04T18:03:02.108 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.110+0000 7f9f35cf7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:02.109 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.110+0000 7f9f35cf7980 20 rados_obj.operate() r=0 bl.length=387 2024-03-04T18:03:02.109 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.110+0000 7f9f35cf7980 10 cache put: name=default.rgw.meta+users.uid+rgw_datacache_user info.flags=0x11 2024-03-04T18:03:02.109 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.110+0000 7f9f35cf7980 10 adding default.rgw.meta+users.uid+rgw_datacache_user to cache LRU end 2024-03-04T18:03:02.109 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.110+0000 7f9f35cf7980 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+rgw_datacache_user 2024-03-04T18:03:02.110 INFO:tasks.workunit.client.0.smithi137.stderr:could not create user: unable to parse parameters, duplicate key provided 2024-03-04T18:03:02.110 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.110+0000 7f9de9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:02.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.114+0000 7f9f35cf7980 20 remove_watcher() i=0 2024-03-04T18:03:02.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.114+0000 7f9f35cf7980 2 removed watcher, disabling cache 2024-03-04T18:03:02.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.118+0000 7f9f35cf7980 20 remove_watcher() i=1 2024-03-04T18:03:02.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.118+0000 7f9f35cf7980 20 remove_watcher() i=2 2024-03-04T18:03:02.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.122+0000 7f9f35cf7980 20 remove_watcher() i=3 2024-03-04T18:03:02.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.122+0000 7f9f35cf7980 20 remove_watcher() i=4 2024-03-04T18:03:02.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.126+0000 7f9f35cf7980 20 remove_watcher() i=5 2024-03-04T18:03:02.128 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.126+0000 7f9f35cf7980 20 remove_watcher() i=6 2024-03-04T18:03:02.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:02.130+0000 7f9f35cf7980 20 remove_watcher() i=7 2024-03-04T18:03:12.066 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:12.067 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:12.179 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:12.284 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:12.284 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:12.394 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:12.394 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:12.468 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:12.469 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:12.560 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:12.560 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:12.637 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: reshard add 2024-03-04T18:03:12.637 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:12.637 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-03-04T18:03:12.670 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:12.670 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:12.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.714+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.714+0000 7ff6848bf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:12.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.714+0000 7ff6848bf980 20 realm 2024-03-04T18:03:12.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.714+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.718+0000 7ff6848bf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:12.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.718+0000 7ff6848bf980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:12.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.718+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.718+0000 7ff6848bf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:12.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.718+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.718+0000 7ff6848bf980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:12.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.718+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.718+0000 7ff6848bf980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:12.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.718+0000 7ff6848bf980 20 searching for the correct realm 2024-03-04T18:03:12.723 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.722+0000 7ff682bd8640 0 -- 172.21.15.137:0/3420731248 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7ff67c0f78d0 msgr2=0x7ff67c10d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:12.937 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.938+0000 7ff682bd8640 0 -- 172.21.15.137:0/3420731248 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55abafd85a10 msgr2=0x55abafd87ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:12.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.946+0000 7ff6848bf980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:12.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.946+0000 7ff6848bf980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:12.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.946+0000 7ff6848bf980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:12.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.946+0000 7ff6848bf980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:12.947 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.946+0000 7ff6848bf980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:12.947 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.946+0000 7ff6848bf980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:12.947 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.946+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.947 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.946+0000 7ff6848bf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:12.947 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.946+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.947 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.946+0000 7ff6848bf980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:12.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.946+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:12.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 20 zone default found 2024-03-04T18:03:12.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 4 Realm: () 2024-03-04T18:03:12.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:12.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:12.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:12.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 20 zonegroup default 2024-03-04T18:03:12.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:12.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.950 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:12.950 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:12.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 20 started sync module instance, tier type = 2024-03-04T18:03:12.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.950+0000 7ff6848bf980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:12.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.974+0000 7ff6848bf980 20 add_watcher() i=0 2024-03-04T18:03:12.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.974+0000 7ff6848bf980 20 add_watcher() i=1 2024-03-04T18:03:12.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.974+0000 7ff6848bf980 20 add_watcher() i=2 2024-03-04T18:03:12.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.974+0000 7ff6848bf980 20 add_watcher() i=3 2024-03-04T18:03:12.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.974+0000 7ff6848bf980 20 add_watcher() i=4 2024-03-04T18:03:12.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.974+0000 7ff6848bf980 20 add_watcher() i=5 2024-03-04T18:03:12.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.974+0000 7ff6848bf980 20 add_watcher() i=6 2024-03-04T18:03:12.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.978+0000 7ff6848bf980 20 add_watcher() i=7 2024-03-04T18:03:12.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.978+0000 7ff6848bf980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:12.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.982+0000 7ff538fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:12.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.982+0000 7ff6848bf980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:12.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.982+0000 7ff6848bf980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:12.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.982+0000 7ff6848bf980 5 note: GC not initialized 2024-03-04T18:03:12.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.982+0000 7ff4b37fe640 20 reqs_thread_entry: start 2024-03-04T18:03:12.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.982+0000 7ff6848bf980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:12.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.982+0000 7ff6848bf980 20 Filter name: none 2024-03-04T18:03:12.983 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.982+0000 7ff4b17fa640 20 reqs_thread_entry: start 2024-03-04T18:03:12.983 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.982+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.983 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.982+0000 7ff6848bf980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:03:12.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.982+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.985 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.986+0000 7ff6848bf980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:03:12.985 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.986+0000 7ff6848bf980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:12.985 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.986+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.986+0000 7ff6848bf980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:03:12.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.986+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.987 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.986+0000 7ff6848bf980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:03:12.987 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.986+0000 7ff6848bf980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:12.987 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.986+0000 7ff6848bf980 20 rados->read ofs=0 len=0 2024-03-04T18:03:12.988 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.990+0000 7ff6848bf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:12.990 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.990+0000 7ff538fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:12.995 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.994+0000 7ff6848bf980 20 remove_watcher() i=0 2024-03-04T18:03:12.995 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.994+0000 7ff6848bf980 2 removed watcher, disabling cache 2024-03-04T18:03:12.997 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.998+0000 7ff6848bf980 20 remove_watcher() i=1 2024-03-04T18:03:12.999 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:12.998+0000 7ff6848bf980 20 remove_watcher() i=2 2024-03-04T18:03:13.002 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.002+0000 7ff6848bf980 20 remove_watcher() i=3 2024-03-04T18:03:13.004 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.006+0000 7ff6848bf980 20 remove_watcher() i=4 2024-03-04T18:03:13.007 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.006+0000 7ff6848bf980 20 remove_watcher() i=5 2024-03-04T18:03:13.009 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.010+0000 7ff6848bf980 20 remove_watcher() i=6 2024-03-04T18:03:13.013 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.014+0000 7ff6848bf980 20 remove_watcher() i=7 2024-03-04T18:03:13.037 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:13.038 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:03:13.038 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: a-bucket id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 num_objects: 0 size_kb: 0 num_shards: 1 2024-03-04T18:03:13.038 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:13.038 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 2 2024-03-04T18:03:13.071 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:13.072 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:13.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.114+0000 7f58ab135980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.118+0000 7f58ab135980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.118+0000 7f58ab135980 20 realm 2024-03-04T18:03:13.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.118+0000 7f58ab135980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.118+0000 7f58ab135980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.118+0000 7f58ab135980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:13.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.118+0000 7f58ab135980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.118+0000 7f58ab135980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.118+0000 7f58ab135980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.118+0000 7f58ab135980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:13.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.118+0000 7f58ab135980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.122+0000 7f58ab135980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:13.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.122+0000 7f58ab135980 20 searching for the correct realm 2024-03-04T18:03:13.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.126+0000 7f58a944e640 0 -- 172.21.15.137:0/2657567111 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f58a0028960 msgr2=0x7f58a0038420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:13.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.342+0000 7f58a944e640 0 -- 172.21.15.137:0/2657567111 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55cdca920870 msgr2=0x55cdca922d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:13.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.350+0000 7f58ab135980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:13.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.350+0000 7f58ab135980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:13.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.350+0000 7f58ab135980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:13.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.350+0000 7f58ab135980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:13.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.350+0000 7f58ab135980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:13.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.350+0000 7f58ab135980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:13.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.350+0000 7f58ab135980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:13.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:13.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 20 zone default found 2024-03-04T18:03:13.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 4 Realm: () 2024-03-04T18:03:13.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:13.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:13.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:13.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 20 zonegroup default 2024-03-04T18:03:13.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.354+0000 7f58ab135980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.358+0000 7f58ab135980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.358+0000 7f58ab135980 20 started sync module instance, tier type = 2024-03-04T18:03:13.356 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.358+0000 7f58ab135980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:13.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.382+0000 7f58ab135980 20 add_watcher() i=0 2024-03-04T18:03:13.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.382+0000 7f58ab135980 20 add_watcher() i=1 2024-03-04T18:03:13.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.382+0000 7f58ab135980 20 add_watcher() i=2 2024-03-04T18:03:13.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.382+0000 7f58ab135980 20 add_watcher() i=3 2024-03-04T18:03:13.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.382+0000 7f58ab135980 20 add_watcher() i=4 2024-03-04T18:03:13.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.382+0000 7f58ab135980 20 add_watcher() i=5 2024-03-04T18:03:13.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.382+0000 7f58ab135980 20 add_watcher() i=6 2024-03-04T18:03:13.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.382+0000 7f58ab135980 20 add_watcher() i=7 2024-03-04T18:03:13.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.382+0000 7f58ab135980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:13.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.390+0000 7f575efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:13.390 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.390+0000 7f58ab135980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:13.390 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.390+0000 7f58ab135980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:13.390 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.390+0000 7f58ab135980 5 note: GC not initialized 2024-03-04T18:03:13.391 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.390+0000 7f575dfe3640 20 reqs_thread_entry: start 2024-03-04T18:03:13.391 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.390+0000 7f58ab135980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:13.391 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.390+0000 7f58ab135980 20 Filter name: none 2024-03-04T18:03:13.391 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.390+0000 7f56cf7fe640 20 reqs_thread_entry: start 2024-03-04T18:03:13.391 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.390+0000 7f58ab135980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-03-04T18:03:13.391 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.390+0000 7f58ab135980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.392 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.394+0000 7f58ab135980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:03:13.392 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.394+0000 7f58ab135980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-03-04T18:03:13.392 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.394+0000 7f58ab135980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-03-04T18:03:13.392 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.394+0000 7f58ab135980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 : miss 2024-03-04T18:03:13.392 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.394+0000 7f58ab135980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.393 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.394+0000 7f58ab135980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:03:13.393 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.394+0000 7f58ab135980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:03:13.393 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.394+0000 7f58ab135980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:03:13.393 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.394+0000 7f58ab135980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:13.393 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.394+0000 7f58ab135980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:03:13.393 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.394+0000 7f58ab135980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 2024-03-04T18:03:13.396 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.398+0000 7f575efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:13.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.402+0000 7f58ab135980 20 remove_watcher() i=0 2024-03-04T18:03:13.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.402+0000 7f58ab135980 2 removed watcher, disabling cache 2024-03-04T18:03:13.403 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.402+0000 7f58ab135980 20 remove_watcher() i=1 2024-03-04T18:03:13.406 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.406+0000 7f58ab135980 20 remove_watcher() i=2 2024-03-04T18:03:13.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.410+0000 7f58ab135980 20 remove_watcher() i=3 2024-03-04T18:03:13.410 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.410+0000 7f58ab135980 20 remove_watcher() i=4 2024-03-04T18:03:13.412 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.414+0000 7f58ab135980 20 remove_watcher() i=5 2024-03-04T18:03:13.415 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.414+0000 7f58ab135980 20 remove_watcher() i=6 2024-03-04T18:03:13.418 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.418+0000 7f58ab135980 20 remove_watcher() i=7 2024-03-04T18:03:13.438 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin reshard list 2024-03-04T18:03:13.466 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:13.467 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:13.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.510+0000 7f231987d980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.513 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.514+0000 7f231987d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.513 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.514+0000 7f231987d980 20 realm 2024-03-04T18:03:13.513 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.514+0000 7f231987d980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.513 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.514+0000 7f231987d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.513 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.514+0000 7f231987d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:13.514 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.514+0000 7f231987d980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.514 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.514+0000 7f231987d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.514 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.514+0000 7f231987d980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.515 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.514+0000 7f231987d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:13.515 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.514+0000 7f231987d980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.516 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.518+0000 7f231987d980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:13.516 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.518+0000 7f231987d980 20 searching for the correct realm 2024-03-04T18:03:13.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.522+0000 7f2317b96640 0 -- 172.21.15.137:0/853215570 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f23081100b0 msgr2=0x7f23081260b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:13.737 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.738+0000 7f2317b96640 0 -- 172.21.15.137:0/853215570 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55ee675291f0 msgr2=0x55ee6752b6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:13.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.746+0000 7f231987d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:13.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.746+0000 7f231987d980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:13.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.746+0000 7f231987d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:13.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.746+0000 7f231987d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:13.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.746+0000 7f231987d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:13.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.746+0000 7f231987d980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:13.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.749 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.749 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:13.749 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:13.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 20 zone default found 2024-03-04T18:03:13.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 4 Realm: () 2024-03-04T18:03:13.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:13.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:13.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:13.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 20 zonegroup default 2024-03-04T18:03:13.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.750+0000 7f231987d980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.754+0000 7f231987d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.754+0000 7f231987d980 20 started sync module instance, tier type = 2024-03-04T18:03:13.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.754+0000 7f231987d980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:13.776 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.774+0000 7f231987d980 20 add_watcher() i=0 2024-03-04T18:03:13.776 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.774+0000 7f231987d980 20 add_watcher() i=1 2024-03-04T18:03:13.776 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.774+0000 7f231987d980 20 add_watcher() i=2 2024-03-04T18:03:13.776 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.774+0000 7f231987d980 20 add_watcher() i=3 2024-03-04T18:03:13.776 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.774+0000 7f231987d980 20 add_watcher() i=4 2024-03-04T18:03:13.776 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.774+0000 7f231987d980 20 add_watcher() i=5 2024-03-04T18:03:13.776 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.774+0000 7f231987d980 20 add_watcher() i=6 2024-03-04T18:03:13.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.778+0000 7f231987d980 20 add_watcher() i=7 2024-03-04T18:03:13.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.778+0000 7f231987d980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:13.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.782+0000 7f21cdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:13.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.782+0000 7f231987d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:13.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.782+0000 7f231987d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:13.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.782+0000 7f231987d980 5 note: GC not initialized 2024-03-04T18:03:13.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.782+0000 7f21ccfe1640 20 reqs_thread_entry: start 2024-03-04T18:03:13.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.786+0000 7f231987d980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:13.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.786+0000 7f231987d980 20 Filter name: none 2024-03-04T18:03:13.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.786+0000 7f21467fc640 20 reqs_thread_entry: start 2024-03-04T18:03:13.798 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.798+0000 7f21cdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:13.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.802+0000 7f231987d980 20 remove_watcher() i=0 2024-03-04T18:03:13.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.802+0000 7f231987d980 2 removed watcher, disabling cache 2024-03-04T18:03:13.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.806+0000 7f231987d980 20 remove_watcher() i=1 2024-03-04T18:03:13.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.806+0000 7f231987d980 20 remove_watcher() i=2 2024-03-04T18:03:13.809 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.810+0000 7f231987d980 20 remove_watcher() i=3 2024-03-04T18:03:13.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.810+0000 7f231987d980 20 remove_watcher() i=4 2024-03-04T18:03:13.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.814+0000 7f231987d980 20 remove_watcher() i=5 2024-03-04T18:03:13.816 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.818+0000 7f231987d980 20 remove_watcher() i=6 2024-03-04T18:03:13.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.818+0000 7f231987d980 20 remove_watcher() i=7 2024-03-04T18:03:13.836 INFO:tasks.workunit.client.0.smithi137.stderr:bucket name a-bucket 2024-03-04T18:03:13.836 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: reshard process 2024-03-04T18:03:13.836 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:13.836 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin reshard process 2024-03-04T18:03:13.864 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:13.864 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:13.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.906+0000 7feb11cf8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.910+0000 7feb11cf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.910+0000 7feb11cf8980 20 realm 2024-03-04T18:03:13.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.910+0000 7feb11cf8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.910+0000 7feb11cf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.910+0000 7feb11cf8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:13.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.910+0000 7feb11cf8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.910+0000 7feb11cf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:13.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.910+0000 7feb11cf8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.910+0000 7feb11cf8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:13.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.910+0000 7feb11cf8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:13.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.910+0000 7feb11cf8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:13.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.910+0000 7feb11cf8980 20 searching for the correct realm 2024-03-04T18:03:13.916 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:13.918+0000 7feb10011640 0 -- 172.21.15.137:0/2563609448 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7feb08060050 msgr2=0x7feb0805a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:14.133 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.134+0000 7feb10011640 0 -- 172.21.15.137:0/2563609448 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55c438b909f0 msgr2=0x55c438b92eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:14.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.142+0000 7feb11cf8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:14.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.142+0000 7feb11cf8980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:14.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.142+0000 7feb11cf8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:14.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.142+0000 7feb11cf8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:14.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.142+0000 7feb11cf8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:14.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.142+0000 7feb11cf8980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:14.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.142+0000 7feb11cf8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:14.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.142+0000 7feb11cf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:14.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.142+0000 7feb11cf8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:14.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.146+0000 7feb11cf8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:14.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.146+0000 7feb11cf8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:14.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.146+0000 7feb11cf8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:14.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.146+0000 7feb11cf8980 20 zone default found 2024-03-04T18:03:14.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.146+0000 7feb11cf8980 4 Realm: () 2024-03-04T18:03:14.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.146+0000 7feb11cf8980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:14.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.146+0000 7feb11cf8980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:14.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.146+0000 7feb11cf8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:14.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.146+0000 7feb11cf8980 20 zonegroup default 2024-03-04T18:03:14.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.146+0000 7feb11cf8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:14.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.146+0000 7feb11cf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:14.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.146+0000 7feb11cf8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:14.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.146+0000 7feb11cf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:14.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.146+0000 7feb11cf8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:14.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.150+0000 7feb11cf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:14.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.150+0000 7feb11cf8980 20 started sync module instance, tier type = 2024-03-04T18:03:14.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.150+0000 7feb11cf8980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:14.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.170+0000 7feb11cf8980 20 add_watcher() i=0 2024-03-04T18:03:14.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.170+0000 7feb11cf8980 20 add_watcher() i=1 2024-03-04T18:03:14.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.170+0000 7feb11cf8980 20 add_watcher() i=2 2024-03-04T18:03:14.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.170+0000 7feb11cf8980 20 add_watcher() i=3 2024-03-04T18:03:14.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.170+0000 7feb11cf8980 20 add_watcher() i=4 2024-03-04T18:03:14.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.170+0000 7feb11cf8980 20 add_watcher() i=5 2024-03-04T18:03:14.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.170+0000 7feb11cf8980 20 add_watcher() i=6 2024-03-04T18:03:14.173 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.174+0000 7feb11cf8980 20 add_watcher() i=7 2024-03-04T18:03:14.173 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.174+0000 7feb11cf8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:14.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.178+0000 7fe9c67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:14.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.178+0000 7feb11cf8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:14.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.178+0000 7feb11cf8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:14.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.178+0000 7feb11cf8980 5 note: GC not initialized 2024-03-04T18:03:14.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.178+0000 7fe9c57e2640 20 reqs_thread_entry: start 2024-03-04T18:03:14.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.178+0000 7feb11cf8980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:14.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.178+0000 7feb11cf8980 20 Filter name: none 2024-03-04T18:03:14.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.178+0000 7fe9be7fc640 20 reqs_thread_entry: start 2024-03-04T18:03:14.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.178+0000 7feb11cf8980 20 processing logshard = reshard.0000000000 2024-03-04T18:03:14.183 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.182+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000000 , ret = 0 2024-03-04T18:03:14.183 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.182+0000 7feb11cf8980 20 processing logshard = reshard.0000000001 2024-03-04T18:03:14.188 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.190+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000001 , ret = 0 2024-03-04T18:03:14.188 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.190+0000 7feb11cf8980 20 processing logshard = reshard.0000000002 2024-03-04T18:03:14.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.194+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000002 , ret = 0 2024-03-04T18:03:14.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.194+0000 7feb11cf8980 20 processing logshard = reshard.0000000003 2024-03-04T18:03:14.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.198+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000003 , ret = 0 2024-03-04T18:03:14.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.198+0000 7feb11cf8980 20 processing logshard = reshard.0000000004 2024-03-04T18:03:14.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.202+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000004 , ret = 0 2024-03-04T18:03:14.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.202+0000 7feb11cf8980 20 processing logshard = reshard.0000000005 2024-03-04T18:03:14.207 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.206+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000005 , ret = 0 2024-03-04T18:03:14.208 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.206+0000 7feb11cf8980 20 processing logshard = reshard.0000000006 2024-03-04T18:03:14.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.210+0000 7feb11cf8980 20 process_entry resharding a-bucket 2024-03-04T18:03:14.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.210+0000 7feb11cf8980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-03-04T18:03:14.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.210+0000 7feb11cf8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:14.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.214+0000 7feb11cf8980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:03:14.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.214+0000 7feb11cf8980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2024-03-04T18:03:14.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.214+0000 7feb11cf8980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-03-04T18:03:14.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.214+0000 7feb11cf8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 : miss 2024-03-04T18:03:14.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.214+0000 7feb11cf8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:14.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.214+0000 7feb11cf8980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:03:14.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.214+0000 7feb11cf8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:03:14.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.214+0000 7feb11cf8980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:03:14.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.214+0000 7feb11cf8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:14.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.214+0000 7feb11cf8980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:03:14.214 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.214+0000 7feb11cf8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 2024-03-04T18:03:14.227 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.226+0000 7feb11cf8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:03:14.227 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.226+0000 7feb11cf8980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:03:14.227 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.226+0000 7feb11cf8980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:03:14.227 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.226+0000 7feb11cf8980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18, ofs0, ns] 2024-03-04T18:03:14.227 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.226+0000 7fe9eeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627803 cookie 94301253625648 notifier 5714 bl.length()=861 2024-03-04T18:03:14.228 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.226+0000 7fe9eeffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:03:14.228 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.226+0000 7fe9eeffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:03:14.228 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.226+0000 7fe9eeffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:03:14.231 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.230+0000 7feb11cf8980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): orig_sources= new_sources= 2024-03-04T18:03:14.231 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.230+0000 7feb11cf8980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): potential sources added=[] removed=[] 2024-03-04T18:03:14.232 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.230+0000 7feb11cf8980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): orig_dests= new_dests= 2024-03-04T18:03:14.232 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.230+0000 7feb11cf8980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): potential dests added=[] removed=[] 2024-03-04T18:03:14.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.234+0000 7feb11cf8980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.1.0 2024-03-04T18:03:14.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.234+0000 7feb11cf8980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.1.0 2024-03-04T18:03:14.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.234+0000 7feb11cf8980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.1.1 2024-03-04T18:03:14.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.234+0000 7feb11cf8980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.1.1 2024-03-04T18:03:14.235 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.234+0000 7feb11cf8980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.0 2024-03-04T18:03:14.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.238+0000 7feb11cf8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 : hit (requested=0x17, cached=0x17) 2024-03-04T18:03:14.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.238+0000 7feb11cf8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 2024-03-04T18:03:14.240 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.242+0000 7feb11cf8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:03:14.240 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.242+0000 7feb11cf8980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:03:14.240 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.242+0000 7feb11cf8980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:03:14.240 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.242+0000 7feb11cf8980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18, ofs0, ns] 2024-03-04T18:03:14.241 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.242+0000 7fe9ee7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627804 cookie 94301253625648 notifier 5714 bl.length()=829 2024-03-04T18:03:14.241 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.242+0000 7fe9ee7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:03:14.241 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.242+0000 7fe9ee7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:03:14.241 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.242+0000 7fe9ee7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:03:14.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.246+0000 7feb11cf8980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): orig_sources= new_sources= 2024-03-04T18:03:14.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.246+0000 7feb11cf8980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): potential sources added=[] removed=[] 2024-03-04T18:03:14.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.246+0000 7feb11cf8980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): orig_dests= new_dests= 2024-03-04T18:03:14.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.246+0000 7feb11cf8980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): potential dests added=[] removed=[] 2024-03-04T18:03:14.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.246+0000 7feb11cf8980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-03-04T18:03:14.251 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.250+0000 7feb11cf8980 20 process_entry removing reshard queue entry for bucket a-bucket 2024-03-04T18:03:14.256 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.258+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000006 , ret = 0 2024-03-04T18:03:14.256 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.258+0000 7feb11cf8980 20 processing logshard = reshard.0000000007 2024-03-04T18:03:14.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.262+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000007 , ret = 0 2024-03-04T18:03:14.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.262+0000 7feb11cf8980 20 processing logshard = reshard.0000000008 2024-03-04T18:03:14.269 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.270+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000008 , ret = 0 2024-03-04T18:03:14.269 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.270+0000 7feb11cf8980 20 processing logshard = reshard.0000000009 2024-03-04T18:03:14.275 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.274+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000009 , ret = 0 2024-03-04T18:03:14.275 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.274+0000 7feb11cf8980 20 processing logshard = reshard.0000000010 2024-03-04T18:03:14.282 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.282+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000010 , ret = 0 2024-03-04T18:03:14.282 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.282+0000 7feb11cf8980 20 processing logshard = reshard.0000000011 2024-03-04T18:03:14.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.290+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000011 , ret = 0 2024-03-04T18:03:14.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.290+0000 7feb11cf8980 20 processing logshard = reshard.0000000012 2024-03-04T18:03:14.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.298+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000012 , ret = 0 2024-03-04T18:03:14.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.298+0000 7feb11cf8980 20 processing logshard = reshard.0000000013 2024-03-04T18:03:14.303 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.302+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000013 , ret = 0 2024-03-04T18:03:14.303 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.302+0000 7feb11cf8980 20 processing logshard = reshard.0000000014 2024-03-04T18:03:14.310 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.310+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000014 , ret = 0 2024-03-04T18:03:14.310 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.310+0000 7feb11cf8980 20 processing logshard = reshard.0000000015 2024-03-04T18:03:14.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.318+0000 7feb11cf8980 20 finish processing logshard = reshard.0000000015 , ret = 0 2024-03-04T18:03:14.318 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.318+0000 7fe9c67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:14.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.322+0000 7feb11cf8980 20 remove_watcher() i=0 2024-03-04T18:03:14.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.322+0000 7feb11cf8980 2 removed watcher, disabling cache 2024-03-04T18:03:14.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.326+0000 7feb11cf8980 20 remove_watcher() i=1 2024-03-04T18:03:14.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.326+0000 7feb11cf8980 20 remove_watcher() i=2 2024-03-04T18:03:14.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.330+0000 7feb11cf8980 20 remove_watcher() i=3 2024-03-04T18:03:14.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.334+0000 7feb11cf8980 20 remove_watcher() i=4 2024-03-04T18:03:14.335 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.334+0000 7feb11cf8980 20 remove_watcher() i=5 2024-03-04T18:03:14.337 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.338+0000 7feb11cf8980 20 remove_watcher() i=6 2024-03-04T18:03:14.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:14.342+0000 7feb11cf8980 20 remove_watcher() i=7 2024-03-04T18:03:19.365 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-03-04T18:03:19.398 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:19.398 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:19.441 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.442+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.442+0000 7f79b26e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:19.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.442+0000 7f79b26e4980 20 realm 2024-03-04T18:03:19.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.442+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.444 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.442+0000 7f79b26e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:19.444 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.442+0000 7f79b26e4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:19.444 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.446+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.444 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.446+0000 7f79b26e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:19.444 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.446+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.446+0000 7f79b26e4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:19.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.446+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.446+0000 7f79b26e4980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:19.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.446+0000 7f79b26e4980 20 searching for the correct realm 2024-03-04T18:03:19.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.450+0000 7f79af9fb640 0 -- 172.21.15.137:0/1823493611 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f79a0017070 msgr2=0x7f79a005c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:19.663 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.662+0000 7f79af9fb640 0 -- 172.21.15.137:0/1823493611 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x558f7f13efd0 msgr2=0x558f7f1413c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:19.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.678+0000 7f79b26e4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:19.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.678+0000 7f79b26e4980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:19.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.678+0000 7f79b26e4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:19.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.678+0000 7f79b26e4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:19.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.678+0000 7f79b26e4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:19.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.678+0000 7f79b26e4980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:19.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.678+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.678+0000 7f79b26e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:19.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.678+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.678+0000 7f79b26e4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:19.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:19.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 20 zone default found 2024-03-04T18:03:19.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 4 Realm: () 2024-03-04T18:03:19.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:19.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:19.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:19.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 20 zonegroup default 2024-03-04T18:03:19.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:19.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:19.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:19.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 20 started sync module instance, tier type = 2024-03-04T18:03:19.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.682+0000 7f79b26e4980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:19.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.706+0000 7f79b26e4980 20 add_watcher() i=0 2024-03-04T18:03:19.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.706+0000 7f79b26e4980 20 add_watcher() i=1 2024-03-04T18:03:19.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.706+0000 7f79b26e4980 20 add_watcher() i=2 2024-03-04T18:03:19.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.706+0000 7f79b26e4980 20 add_watcher() i=3 2024-03-04T18:03:19.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.706+0000 7f79b26e4980 20 add_watcher() i=4 2024-03-04T18:03:19.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.706+0000 7f79b26e4980 20 add_watcher() i=5 2024-03-04T18:03:19.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.706+0000 7f79b26e4980 20 add_watcher() i=6 2024-03-04T18:03:19.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.706+0000 7f79b26e4980 20 add_watcher() i=7 2024-03-04T18:03:19.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.706+0000 7f79b26e4980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:19.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.710+0000 7f7866fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:19.712 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.714+0000 7f79b26e4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:19.712 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.714+0000 7f79b26e4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:19.712 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.714+0000 7f79b26e4980 5 note: GC not initialized 2024-03-04T18:03:19.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.714+0000 7f7865fdb640 20 reqs_thread_entry: start 2024-03-04T18:03:19.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.714+0000 7f79b26e4980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:19.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.714+0000 7f79b26e4980 20 Filter name: none 2024-03-04T18:03:19.714 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.714+0000 7f77d77fe640 20 reqs_thread_entry: start 2024-03-04T18:03:19.714 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.714+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.714 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.714+0000 7f79b26e4980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:03:19.714 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.714+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.714+0000 7f79b26e4980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:03:19.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.714+0000 7f79b26e4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:19.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.718+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.718+0000 7f79b26e4980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:03:19.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.718+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.718+0000 7f79b26e4980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:03:19.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.718+0000 7f79b26e4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:19.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.718+0000 7f79b26e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:19.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.718+0000 7f79b26e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:19.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.722+0000 7f7866fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:19.725 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.726+0000 7f79b26e4980 20 remove_watcher() i=0 2024-03-04T18:03:19.725 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.726+0000 7f79b26e4980 2 removed watcher, disabling cache 2024-03-04T18:03:19.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.726+0000 7f79b26e4980 20 remove_watcher() i=1 2024-03-04T18:03:19.729 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.730+0000 7f79b26e4980 20 remove_watcher() i=2 2024-03-04T18:03:19.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.730+0000 7f79b26e4980 20 remove_watcher() i=3 2024-03-04T18:03:19.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.734+0000 7f79b26e4980 20 remove_watcher() i=4 2024-03-04T18:03:19.736 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.738+0000 7f79b26e4980 20 remove_watcher() i=5 2024-03-04T18:03:19.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.738+0000 7f79b26e4980 20 remove_watcher() i=6 2024-03-04T18:03:19.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:19.742+0000 7f79b26e4980 20 remove_watcher() i=7 2024-03-04T18:03:19.760 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:19.760 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:03:19.760 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: a-bucket id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 num_objects: 0 size_kb: 0 num_shards: 2 2024-03-04T18:03:19.760 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:19.760 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: reshard add non empty bucket 2024-03-04T18:03:19.760 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:19.764 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:19.764 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:19.818 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:19.818 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:19.895 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:19.895 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:19.978 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:19.978 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:20.055 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:20.056 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:20.132 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:20.132 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:20.216 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:20.216 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:20.300 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:20.300 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:20.385 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-03-04T18:03:20.417 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:20.417 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:20.460 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.462+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.462+0000 7fd23d4e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:20.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.462+0000 7fd23d4e4980 20 realm 2024-03-04T18:03:20.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.462+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.462+0000 7fd23d4e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:20.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.462+0000 7fd23d4e4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:20.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.462+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.466+0000 7fd23d4e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:20.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.466+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.466+0000 7fd23d4e4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:20.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.466+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.466+0000 7fd23d4e4980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:20.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.466+0000 7fd23d4e4980 20 searching for the correct realm 2024-03-04T18:03:20.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.470+0000 7fd23b7fd640 0 -- 172.21.15.137:0/1195584208 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fd2340555d0 msgr2=0x7fd234053930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:20.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.686+0000 7fd23b7fd640 0 -- 172.21.15.137:0/1195584208 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x558ee2dadb30 msgr2=0x558ee2dafff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:20.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.694+0000 7fd23d4e4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:20.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.694+0000 7fd23d4e4980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:20.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.694+0000 7fd23d4e4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:20.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.694+0000 7fd23d4e4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:20.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.694+0000 7fd23d4e4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:20.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.694+0000 7fd23d4e4980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:20.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.694+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.694+0000 7fd23d4e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:20.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.694+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:20.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:20.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 20 zone default found 2024-03-04T18:03:20.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 4 Realm: () 2024-03-04T18:03:20.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:20.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:20.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:20.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 20 zonegroup default 2024-03-04T18:03:20.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:20.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:20.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:20.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 20 started sync module instance, tier type = 2024-03-04T18:03:20.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.698+0000 7fd23d4e4980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:20.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.722+0000 7fd23d4e4980 20 add_watcher() i=0 2024-03-04T18:03:20.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.722+0000 7fd23d4e4980 20 add_watcher() i=1 2024-03-04T18:03:20.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.722+0000 7fd23d4e4980 20 add_watcher() i=2 2024-03-04T18:03:20.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.722+0000 7fd23d4e4980 20 add_watcher() i=3 2024-03-04T18:03:20.723 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.722+0000 7fd23d4e4980 20 add_watcher() i=4 2024-03-04T18:03:20.723 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.722+0000 7fd23d4e4980 20 add_watcher() i=5 2024-03-04T18:03:20.723 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.722+0000 7fd23d4e4980 20 add_watcher() i=6 2024-03-04T18:03:20.725 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.726+0000 7fd23d4e4980 20 add_watcher() i=7 2024-03-04T18:03:20.725 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.726+0000 7fd23d4e4980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:20.729 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.730+0000 7fd0f17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:20.730 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.730+0000 7fd23d4e4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:20.730 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.730+0000 7fd23d4e4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:20.730 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.730+0000 7fd23d4e4980 5 note: GC not initialized 2024-03-04T18:03:20.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.730+0000 7fd06bfff640 20 reqs_thread_entry: start 2024-03-04T18:03:20.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.730+0000 7fd23d4e4980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:20.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.730+0000 7fd23d4e4980 20 Filter name: none 2024-03-04T18:03:20.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.730+0000 7fd069ffb640 20 reqs_thread_entry: start 2024-03-04T18:03:20.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.730+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.734+0000 7fd23d4e4980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:03:20.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.734+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.734+0000 7fd23d4e4980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:03:20.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.734+0000 7fd23d4e4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:20.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.734+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.734+0000 7fd23d4e4980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:03:20.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.734+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.735 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.734+0000 7fd23d4e4980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:03:20.735 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.734+0000 7fd23d4e4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:20.736 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.738+0000 7fd23d4e4980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.737 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.738+0000 7fd23d4e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:20.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.738+0000 7fd0f17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:20.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.742+0000 7fd23d4e4980 20 remove_watcher() i=0 2024-03-04T18:03:20.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.742+0000 7fd23d4e4980 2 removed watcher, disabling cache 2024-03-04T18:03:20.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.746+0000 7fd23d4e4980 20 remove_watcher() i=1 2024-03-04T18:03:20.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.746+0000 7fd23d4e4980 20 remove_watcher() i=2 2024-03-04T18:03:20.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.750+0000 7fd23d4e4980 20 remove_watcher() i=3 2024-03-04T18:03:20.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.750+0000 7fd23d4e4980 20 remove_watcher() i=4 2024-03-04T18:03:20.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.754+0000 7fd23d4e4980 20 remove_watcher() i=5 2024-03-04T18:03:20.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.754+0000 7fd23d4e4980 20 remove_watcher() i=6 2024-03-04T18:03:20.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.758+0000 7fd23d4e4980 20 remove_watcher() i=7 2024-03-04T18:03:20.774 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:20.774 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:03:20.774 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: a-bucket id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 num_objects: 8 size_kb: 1 num_shards: 2 2024-03-04T18:03:20.774 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:20.774 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 3 2024-03-04T18:03:20.803 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:20.803 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:20.845 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.846+0000 7f860ec83980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.846+0000 7f860ec83980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:20.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.846+0000 7f860ec83980 20 realm 2024-03-04T18:03:20.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.846+0000 7f860ec83980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.850+0000 7f860ec83980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:20.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.850+0000 7f860ec83980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:20.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.850+0000 7f860ec83980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.850+0000 7f860ec83980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:20.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.850+0000 7f860ec83980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.850 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.850+0000 7f860ec83980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:20.850 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.850+0000 7f860ec83980 20 rados->read ofs=0 len=0 2024-03-04T18:03:20.850 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.850+0000 7f860ec83980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:20.850 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.850+0000 7f860ec83980 20 searching for the correct realm 2024-03-04T18:03:20.856 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:20.858+0000 7f860cf9c640 0 -- 172.21.15.137:0/754373116 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f860405b0b0 msgr2=0x7f86040535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:21.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.074+0000 7f860cf9c640 0 -- 172.21.15.137:0/754373116 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x563046451a50 msgr2=0x563046453f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:21.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.082+0000 7f860ec83980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:21.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.082+0000 7f860ec83980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:21.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.082+0000 7f860ec83980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:21.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.082+0000 7f860ec83980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:21.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.082+0000 7f860ec83980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:21.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.082+0000 7f860ec83980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:21.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.082+0000 7f860ec83980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.082+0000 7f860ec83980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:21.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.082+0000 7f860ec83980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.082+0000 7f860ec83980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:21.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.082+0000 7f860ec83980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:21.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 20 zone default found 2024-03-04T18:03:21.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 4 Realm: () 2024-03-04T18:03:21.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:21.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:21.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:21.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 20 zonegroup default 2024-03-04T18:03:21.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:21.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:21.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:21.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 20 started sync module instance, tier type = 2024-03-04T18:03:21.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.086+0000 7f860ec83980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:21.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.110+0000 7f860ec83980 20 add_watcher() i=0 2024-03-04T18:03:21.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.110+0000 7f860ec83980 20 add_watcher() i=1 2024-03-04T18:03:21.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.110+0000 7f860ec83980 20 add_watcher() i=2 2024-03-04T18:03:21.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.110+0000 7f860ec83980 20 add_watcher() i=3 2024-03-04T18:03:21.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.110+0000 7f860ec83980 20 add_watcher() i=4 2024-03-04T18:03:21.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.110+0000 7f860ec83980 20 add_watcher() i=5 2024-03-04T18:03:21.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.110+0000 7f860ec83980 20 add_watcher() i=6 2024-03-04T18:03:21.113 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.114+0000 7f860ec83980 20 add_watcher() i=7 2024-03-04T18:03:21.113 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.114+0000 7f860ec83980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:21.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.118+0000 7f84c37de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:21.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.118+0000 7f860ec83980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:21.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.118+0000 7f860ec83980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:21.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.118+0000 7f860ec83980 5 note: GC not initialized 2024-03-04T18:03:21.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.118+0000 7f84c27dc640 20 reqs_thread_entry: start 2024-03-04T18:03:21.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.118+0000 7f860ec83980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:21.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.118+0000 7f860ec83980 20 Filter name: none 2024-03-04T18:03:21.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.118+0000 7f84bb7fe640 20 reqs_thread_entry: start 2024-03-04T18:03:21.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.118+0000 7f860ec83980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-03-04T18:03:21.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.118+0000 7f860ec83980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.122+0000 7f860ec83980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:03:21.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.122+0000 7f860ec83980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-03-04T18:03:21.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.122+0000 7f860ec83980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-03-04T18:03:21.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.122+0000 7f860ec83980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 : miss 2024-03-04T18:03:21.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.122+0000 7f860ec83980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.122+0000 7f860ec83980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:03:21.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.122+0000 7f860ec83980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:03:21.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.122+0000 7f860ec83980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:03:21.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.122+0000 7f860ec83980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:21.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.122+0000 7f860ec83980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:03:21.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.122+0000 7f860ec83980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 2024-03-04T18:03:21.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.126+0000 7f84c37de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:21.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.130+0000 7f860ec83980 20 remove_watcher() i=0 2024-03-04T18:03:21.130 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.130+0000 7f860ec83980 2 removed watcher, disabling cache 2024-03-04T18:03:21.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.134+0000 7f860ec83980 20 remove_watcher() i=1 2024-03-04T18:03:21.134 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.134+0000 7f860ec83980 20 remove_watcher() i=2 2024-03-04T18:03:21.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.138+0000 7f860ec83980 20 remove_watcher() i=3 2024-03-04T18:03:21.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.138+0000 7f860ec83980 20 remove_watcher() i=4 2024-03-04T18:03:21.141 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.142+0000 7f860ec83980 20 remove_watcher() i=5 2024-03-04T18:03:21.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.146+0000 7f860ec83980 20 remove_watcher() i=6 2024-03-04T18:03:21.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.146+0000 7f860ec83980 20 remove_watcher() i=7 2024-03-04T18:03:21.165 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin reshard list 2024-03-04T18:03:21.195 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:21.195 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:21.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.238+0000 7f522eda7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.240 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.242+0000 7f522eda7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:21.240 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.242+0000 7f522eda7980 20 realm 2024-03-04T18:03:21.240 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.242+0000 7f522eda7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.241 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.242+0000 7f522eda7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:21.241 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.242+0000 7f522eda7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:21.241 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.242+0000 7f522eda7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.241 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.242+0000 7f522eda7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:21.241 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.242+0000 7f522eda7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.242+0000 7f522eda7980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:21.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.242+0000 7f522eda7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.242+0000 7f522eda7980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:21.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.242+0000 7f522eda7980 20 searching for the correct realm 2024-03-04T18:03:21.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.250+0000 7f522d0c0640 0 -- 172.21.15.137:0/517641650 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f5224062c30 msgr2=0x7f5224073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:21.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.466+0000 7f522d0c0640 0 -- 172.21.15.137:0/517641650 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55f14a878920 msgr2=0x55f14a87ade0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:21.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.474+0000 7f522eda7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:21.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.474+0000 7f522eda7980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:21.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.474+0000 7f522eda7980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:21.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.474+0000 7f522eda7980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:21.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.474+0000 7f522eda7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:21.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.474+0000 7f522eda7980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:21.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.474+0000 7f522eda7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.476 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.474+0000 7f522eda7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:21.476 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.474+0000 7f522eda7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.476 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:21.476 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.476 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:21.477 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 20 zone default found 2024-03-04T18:03:21.477 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 4 Realm: () 2024-03-04T18:03:21.477 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:21.477 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:21.477 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:21.477 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 20 zonegroup default 2024-03-04T18:03:21.477 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.477 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:21.477 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.478 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:21.478 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:21.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 20 started sync module instance, tier type = 2024-03-04T18:03:21.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.478+0000 7f522eda7980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:21.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.502+0000 7f522eda7980 20 add_watcher() i=0 2024-03-04T18:03:21.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.502+0000 7f522eda7980 20 add_watcher() i=1 2024-03-04T18:03:21.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.502+0000 7f522eda7980 20 add_watcher() i=2 2024-03-04T18:03:21.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.502+0000 7f522eda7980 20 add_watcher() i=3 2024-03-04T18:03:21.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.502+0000 7f522eda7980 20 add_watcher() i=4 2024-03-04T18:03:21.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.502+0000 7f522eda7980 20 add_watcher() i=5 2024-03-04T18:03:21.501 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.502+0000 7f522eda7980 20 add_watcher() i=6 2024-03-04T18:03:21.504 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.506+0000 7f522eda7980 20 add_watcher() i=7 2024-03-04T18:03:21.504 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.506+0000 7f522eda7980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:21.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.510+0000 7f50e2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:21.509 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.510+0000 7f522eda7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:21.509 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.510+0000 7f522eda7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:21.509 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.510+0000 7f522eda7980 5 note: GC not initialized 2024-03-04T18:03:21.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.510+0000 7f50e1fe3640 20 reqs_thread_entry: start 2024-03-04T18:03:21.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.510+0000 7f522eda7980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:21.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.510+0000 7f522eda7980 20 Filter name: none 2024-03-04T18:03:21.511 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.510+0000 7f50537fe640 20 reqs_thread_entry: start 2024-03-04T18:03:21.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.522+0000 7f50e2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:21.527 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.526+0000 7f522eda7980 20 remove_watcher() i=0 2024-03-04T18:03:21.527 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.526+0000 7f522eda7980 2 removed watcher, disabling cache 2024-03-04T18:03:21.530 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.530+0000 7f522eda7980 20 remove_watcher() i=1 2024-03-04T18:03:21.532 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.534+0000 7f522eda7980 20 remove_watcher() i=2 2024-03-04T18:03:21.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.534+0000 7f522eda7980 20 remove_watcher() i=3 2024-03-04T18:03:21.537 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.538+0000 7f522eda7980 20 remove_watcher() i=4 2024-03-04T18:03:21.539 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.538+0000 7f522eda7980 20 remove_watcher() i=5 2024-03-04T18:03:21.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.542+0000 7f522eda7980 20 remove_watcher() i=6 2024-03-04T18:03:21.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.546+0000 7f522eda7980 20 remove_watcher() i=7 2024-03-04T18:03:21.566 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: reshard process non empty bucket 2024-03-04T18:03:21.566 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:21.567 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin reshard process 2024-03-04T18:03:21.596 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:21.596 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:21.836 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.838+0000 7f78f698b640 0 --2- 172.21.15.137:0/3125133009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bc2320360 0x563bc23207e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-03-04T18:03:21.845 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.846+0000 7f78f8672980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.847 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.846+0000 7f78f8672980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:21.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.846+0000 7f78f8672980 20 realm 2024-03-04T18:03:21.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.846+0000 7f78f8672980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.850+0000 7f78f8672980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:21.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.850+0000 7f78f8672980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:21.848 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.850+0000 7f78f8672980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.850+0000 7f78f8672980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:21.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.850+0000 7f78f8672980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.850 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.850+0000 7f78f8672980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:21.850 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.850+0000 7f78f8672980 20 rados->read ofs=0 len=0 2024-03-04T18:03:21.851 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.850+0000 7f78f8672980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:21.851 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.850+0000 7f78f8672980 20 searching for the correct realm 2024-03-04T18:03:21.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.854+0000 7f78f698b640 0 -- 172.21.15.137:0/4058894771 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x563bc2347930 msgr2=0x563bc2349db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:21.861 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:21.862+0000 7f78f618a640 0 -- 172.21.15.137:0/4058894771 >> [v2:172.21.15.173:6805/879938785,v1:172.21.15.173:6808/879938785] conn(0x7f78e8087be0 msgr2=0x7f78e8046120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:22.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.082+0000 7f78f8672980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:22.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.082+0000 7f78f8672980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:22.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.082+0000 7f78f8672980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:22.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.082+0000 7f78f8672980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:22.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.082+0000 7f78f8672980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:22.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.082+0000 7f78f8672980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:22.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.082+0000 7f78f8672980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.082+0000 7f78f8672980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:22.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.082+0000 7f78f8672980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.082+0000 7f78f8672980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:22.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.082+0000 7f78f8672980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:22.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 20 zone default found 2024-03-04T18:03:22.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 4 Realm: () 2024-03-04T18:03:22.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:22.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:22.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:22.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 20 zonegroup default 2024-03-04T18:03:22.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:22.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:22.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:22.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 20 started sync module instance, tier type = 2024-03-04T18:03:22.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.086+0000 7f78f8672980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:22.112 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.110+0000 7f78f8672980 20 add_watcher() i=0 2024-03-04T18:03:22.112 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.110+0000 7f78f8672980 20 add_watcher() i=1 2024-03-04T18:03:22.112 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.110+0000 7f78f8672980 20 add_watcher() i=2 2024-03-04T18:03:22.112 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.110+0000 7f78f8672980 20 add_watcher() i=3 2024-03-04T18:03:22.112 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.110+0000 7f78f8672980 20 add_watcher() i=4 2024-03-04T18:03:22.112 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.110+0000 7f78f8672980 20 add_watcher() i=5 2024-03-04T18:03:22.112 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.110+0000 7f78f8672980 20 add_watcher() i=6 2024-03-04T18:03:22.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.114+0000 7f78f8672980 20 add_watcher() i=7 2024-03-04T18:03:22.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.114+0000 7f78f8672980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:22.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.118+0000 7f77acfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:22.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.118+0000 7f78f8672980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:22.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.118+0000 7f78f8672980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:22.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.118+0000 7f78f8672980 5 note: GC not initialized 2024-03-04T18:03:22.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.122+0000 7f77277fe640 20 reqs_thread_entry: start 2024-03-04T18:03:22.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.122+0000 7f78f8672980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:22.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.122+0000 7f78f8672980 20 Filter name: none 2024-03-04T18:03:22.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.122+0000 7f77257fa640 20 reqs_thread_entry: start 2024-03-04T18:03:22.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.122+0000 7f78f8672980 20 processing logshard = reshard.0000000000 2024-03-04T18:03:22.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.126+0000 7f78f8672980 20 finish processing logshard = reshard.0000000000 , ret = 0 2024-03-04T18:03:22.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.126+0000 7f78f8672980 20 processing logshard = reshard.0000000001 2024-03-04T18:03:22.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.130+0000 7f78f8672980 20 finish processing logshard = reshard.0000000001 , ret = 0 2024-03-04T18:03:22.132 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.130+0000 7f78f8672980 20 processing logshard = reshard.0000000002 2024-03-04T18:03:22.136 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.138+0000 7f78f8672980 20 finish processing logshard = reshard.0000000002 , ret = 0 2024-03-04T18:03:22.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.138+0000 7f78f8672980 20 processing logshard = reshard.0000000003 2024-03-04T18:03:22.141 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.142+0000 7f78f8672980 20 finish processing logshard = reshard.0000000003 , ret = 0 2024-03-04T18:03:22.141 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.142+0000 7f78f8672980 20 processing logshard = reshard.0000000004 2024-03-04T18:03:22.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.146+0000 7f78f8672980 20 finish processing logshard = reshard.0000000004 , ret = 0 2024-03-04T18:03:22.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.146+0000 7f78f8672980 20 processing logshard = reshard.0000000005 2024-03-04T18:03:22.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.150+0000 7f78f8672980 20 finish processing logshard = reshard.0000000005 , ret = 0 2024-03-04T18:03:22.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.150+0000 7f78f8672980 20 processing logshard = reshard.0000000006 2024-03-04T18:03:22.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.154+0000 7f78f8672980 20 process_entry resharding a-bucket 2024-03-04T18:03:22.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.154+0000 7f78f8672980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-03-04T18:03:22.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.154+0000 7f78f8672980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.156 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.154+0000 7f78f8672980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:03:22.156 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.154+0000 7f78f8672980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2024-03-04T18:03:22.156 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.154+0000 7f78f8672980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-03-04T18:03:22.156 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.154+0000 7f78f8672980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 : miss 2024-03-04T18:03:22.156 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.154+0000 7f78f8672980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.156 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.158+0000 7f78f8672980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:03:22.156 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.158+0000 7f78f8672980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:03:22.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.158+0000 7f78f8672980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:03:22.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.158+0000 7f78f8672980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:22.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.158+0000 7f78f8672980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:03:22.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.158+0000 7f78f8672980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 2024-03-04T18:03:22.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.170+0000 7f78f8672980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:03:22.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.170+0000 7f78f8672980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:03:22.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.170+0000 7f78f8672980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:03:22.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.170+0000 7f78f8672980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18, ofs0, ns] 2024-03-04T18:03:22.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.170+0000 7f77d9ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627805 cookie 94814661358416 notifier 5795 bl.length()=861 2024-03-04T18:03:22.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.170+0000 7f77d9ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:03:22.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.170+0000 7f77d9ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:03:22.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.170+0000 7f77d9ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:03:22.173 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.174+0000 7f78f8672980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): orig_sources= new_sources= 2024-03-04T18:03:22.173 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.174+0000 7f78f8672980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): potential sources added=[] removed=[] 2024-03-04T18:03:22.174 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.174+0000 7f78f8672980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): orig_dests= new_dests= 2024-03-04T18:03:22.174 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.174+0000 7f78f8672980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): potential dests added=[] removed=[] 2024-03-04T18:03:22.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.174+0000 7f78f8672980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.0 2024-03-04T18:03:22.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.174+0000 7f78f8672980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.0 2024-03-04T18:03:22.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.174+0000 7f78f8672980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.1 2024-03-04T18:03:22.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.174+0000 7f78f8672980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.1 2024-03-04T18:03:22.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.174+0000 7f78f8672980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.2 2024-03-04T18:03:22.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.174+0000 7f78f8672980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.2 2024-03-04T18:03:22.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.174+0000 7f78f8672980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.1.0 2024-03-04T18:03:22.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.174+0000 7f78f8672980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.1.0 2024-03-04T18:03:22.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.178+0000 7f78f8672980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.1.1 2024-03-04T18:03:22.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.178+0000 7f78f8672980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.1.1 2024-03-04T18:03:22.182 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.182+0000 7f78f8672980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 : hit (requested=0x17, cached=0x17) 2024-03-04T18:03:22.182 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.182+0000 7f78f8672980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 2024-03-04T18:03:22.185 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.186+0000 7f78f8672980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:03:22.185 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.186+0000 7f78f8672980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:03:22.185 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.186+0000 7f78f8672980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:03:22.185 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.186+0000 7f78f8672980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18, ofs0, ns] 2024-03-04T18:03:22.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.186+0000 7f77d97fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627806 cookie 94814661358416 notifier 5795 bl.length()=829 2024-03-04T18:03:22.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.186+0000 7f77d97fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:03:22.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.186+0000 7f77d97fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:03:22.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.186+0000 7f77d97fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:03:22.188 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.190+0000 7f78f8672980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): orig_sources= new_sources= 2024-03-04T18:03:22.188 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.190+0000 7f78f8672980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): potential sources added=[] removed=[] 2024-03-04T18:03:22.188 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.190+0000 7f78f8672980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): orig_dests= new_dests= 2024-03-04T18:03:22.189 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.190+0000 7f78f8672980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): potential dests added=[] removed=[] 2024-03-04T18:03:22.191 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.190+0000 7f78f8672980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-03-04T18:03:22.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.194+0000 7f78f8672980 20 process_entry removing reshard queue entry for bucket a-bucket 2024-03-04T18:03:22.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.198+0000 7f78f8672980 20 finish processing logshard = reshard.0000000006 , ret = 0 2024-03-04T18:03:22.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.198+0000 7f78f8672980 20 processing logshard = reshard.0000000007 2024-03-04T18:03:22.205 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.206+0000 7f78f8672980 20 finish processing logshard = reshard.0000000007 , ret = 0 2024-03-04T18:03:22.205 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.206+0000 7f78f8672980 20 processing logshard = reshard.0000000008 2024-03-04T18:03:22.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.210+0000 7f78f8672980 20 finish processing logshard = reshard.0000000008 , ret = 0 2024-03-04T18:03:22.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.210+0000 7f78f8672980 20 processing logshard = reshard.0000000009 2024-03-04T18:03:22.217 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.218+0000 7f78f8672980 20 finish processing logshard = reshard.0000000009 , ret = 0 2024-03-04T18:03:22.217 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.218+0000 7f78f8672980 20 processing logshard = reshard.0000000010 2024-03-04T18:03:22.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.226+0000 7f78f8672980 20 finish processing logshard = reshard.0000000010 , ret = 0 2024-03-04T18:03:22.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.226+0000 7f78f8672980 20 processing logshard = reshard.0000000011 2024-03-04T18:03:22.231 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.230+0000 7f78f8672980 20 finish processing logshard = reshard.0000000011 , ret = 0 2024-03-04T18:03:22.231 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.230+0000 7f78f8672980 20 processing logshard = reshard.0000000012 2024-03-04T18:03:22.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.238+0000 7f78f8672980 20 finish processing logshard = reshard.0000000012 , ret = 0 2024-03-04T18:03:22.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.238+0000 7f78f8672980 20 processing logshard = reshard.0000000013 2024-03-04T18:03:22.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.242+0000 7f78f8672980 20 finish processing logshard = reshard.0000000013 , ret = 0 2024-03-04T18:03:22.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.242+0000 7f78f8672980 20 processing logshard = reshard.0000000014 2024-03-04T18:03:22.250 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.250+0000 7f78f8672980 20 finish processing logshard = reshard.0000000014 , ret = 0 2024-03-04T18:03:22.250 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.250+0000 7f78f8672980 20 processing logshard = reshard.0000000015 2024-03-04T18:03:22.256 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.258+0000 7f78f8672980 20 finish processing logshard = reshard.0000000015 , ret = 0 2024-03-04T18:03:22.258 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.258+0000 7f77acfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:22.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.262+0000 7f78f8672980 20 remove_watcher() i=0 2024-03-04T18:03:22.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.262+0000 7f78f8672980 2 removed watcher, disabling cache 2024-03-04T18:03:22.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.266+0000 7f78f8672980 20 remove_watcher() i=1 2024-03-04T18:03:22.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.266+0000 7f78f8672980 20 remove_watcher() i=2 2024-03-04T18:03:22.270 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.270+0000 7f78f8672980 20 remove_watcher() i=3 2024-03-04T18:03:22.272 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.274+0000 7f78f8672980 20 remove_watcher() i=4 2024-03-04T18:03:22.274 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.274+0000 7f78f8672980 20 remove_watcher() i=5 2024-03-04T18:03:22.277 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.278+0000 7f78f8672980 20 remove_watcher() i=6 2024-03-04T18:03:22.280 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.282+0000 7f78f8672980 20 remove_watcher() i=7 2024-03-04T18:03:22.304 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-03-04T18:03:22.335 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:22.335 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:22.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.378+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.382+0000 7f584213a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:22.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.382+0000 7f584213a980 20 realm 2024-03-04T18:03:22.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.382+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.382+0000 7f584213a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:22.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.382+0000 7f584213a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:22.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.382+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.382+0000 7f584213a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:22.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.382+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.384 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.382+0000 7f584213a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:22.384 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.382+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.384 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.386+0000 7f584213a980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:22.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.386+0000 7f584213a980 20 searching for the correct realm 2024-03-04T18:03:22.390 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.390+0000 7f5840453640 0 -- 172.21.15.137:0/2674642917 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f58380608c0 msgr2=0x7f5838059970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:22.605 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.606+0000 7f5840453640 0 -- 172.21.15.137:0/2674642917 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x562ed1d192f0 msgr2=0x562ed1d1b6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:22.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.614+0000 7f584213a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:22.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.614+0000 7f584213a980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:22.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.614+0000 7f584213a980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:22.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.614+0000 7f584213a980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:22.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.614+0000 7f584213a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:22.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.614+0000 7f584213a980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:22.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.614+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.614+0000 7f584213a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:22.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.614+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:22.617 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.617 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:22.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 20 zone default found 2024-03-04T18:03:22.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 4 Realm: () 2024-03-04T18:03:22.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:22.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:22.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:22.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 20 zonegroup default 2024-03-04T18:03:22.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:22.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:22.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.620 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:22.620 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 20 started sync module instance, tier type = 2024-03-04T18:03:22.620 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.618+0000 7f584213a980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:22.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.642+0000 7f584213a980 20 add_watcher() i=0 2024-03-04T18:03:22.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.642+0000 7f584213a980 20 add_watcher() i=1 2024-03-04T18:03:22.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.642+0000 7f584213a980 20 add_watcher() i=2 2024-03-04T18:03:22.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.642+0000 7f584213a980 20 add_watcher() i=3 2024-03-04T18:03:22.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.642+0000 7f584213a980 20 add_watcher() i=4 2024-03-04T18:03:22.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.642+0000 7f584213a980 20 add_watcher() i=5 2024-03-04T18:03:22.643 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.642+0000 7f584213a980 20 add_watcher() i=6 2024-03-04T18:03:22.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.646+0000 7f584213a980 20 add_watcher() i=7 2024-03-04T18:03:22.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.646+0000 7f584213a980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:22.649 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.650+0000 7f56f5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:22.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.650+0000 7f584213a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:22.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.650+0000 7f584213a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:22.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.650+0000 7f584213a980 5 note: GC not initialized 2024-03-04T18:03:22.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.650+0000 7f56f4fe1640 20 reqs_thread_entry: start 2024-03-04T18:03:22.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.650+0000 7f584213a980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:22.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.650+0000 7f584213a980 20 Filter name: none 2024-03-04T18:03:22.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.650+0000 7f566e7fc640 20 reqs_thread_entry: start 2024-03-04T18:03:22.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.650+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.650+0000 7f584213a980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:03:22.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.650+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.650+0000 7f584213a980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:03:22.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.650+0000 7f584213a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:22.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.650+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.654+0000 7f584213a980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:03:22.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.654+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.653 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.654+0000 7f584213a980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:03:22.653 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.654+0000 7f584213a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:22.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.654+0000 7f584213a980 20 rados->read ofs=0 len=0 2024-03-04T18:03:22.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.654+0000 7f584213a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:22.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.658+0000 7f56f5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:22.660 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.658+0000 7f584213a980 20 remove_watcher() i=0 2024-03-04T18:03:22.660 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.658+0000 7f584213a980 2 removed watcher, disabling cache 2024-03-04T18:03:22.662 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.662+0000 7f584213a980 20 remove_watcher() i=1 2024-03-04T18:03:22.664 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.666+0000 7f584213a980 20 remove_watcher() i=2 2024-03-04T18:03:22.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.666+0000 7f584213a980 20 remove_watcher() i=3 2024-03-04T18:03:22.668 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.670+0000 7f584213a980 20 remove_watcher() i=4 2024-03-04T18:03:22.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.670+0000 7f584213a980 20 remove_watcher() i=5 2024-03-04T18:03:22.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.674+0000 7f584213a980 20 remove_watcher() i=6 2024-03-04T18:03:22.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:22.678+0000 7f584213a980 20 remove_watcher() i=7 2024-03-04T18:03:22.691 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:22.691 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:03:22.691 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: a-bucket id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 num_objects: 8 size_kb: 1 num_shards: 3 2024-03-04T18:03:22.691 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:22.691 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: reshard bucket with EIO injected at set_target_layout 2024-03-04T18:03:22.691 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:22.692 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:22.692 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:22.757 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:22.757 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:22.836 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:22.836 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:22.917 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:22.917 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:22.997 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:22.997 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:23.082 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:23.082 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:23.146 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:23.146 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:23.228 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:23.228 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:23.313 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:23.313 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:23.394 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:23.394 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:23.455 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:23.456 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:23.543 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:23.543 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:23.630 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:23.630 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:23.714 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:23.714 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:23.795 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:23.795 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:23.880 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:23.881 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:23.961 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:23.961 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:24.046 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:24.046 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:24.131 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:24.131 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:24.214 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:24.214 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:24.300 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:24.300 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:24.387 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:24.387 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:24.468 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-03-04T18:03:24.501 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:24.501 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:24.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.754+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:24.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.758+0000 7f75de7ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:24.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.758+0000 7f75de7ef980 20 realm 2024-03-04T18:03:24.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.758+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:24.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.758+0000 7f75de7ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:24.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.758+0000 7f75de7ef980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:24.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.758+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:24.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.758+0000 7f75de7ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:24.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.758+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:24.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.758+0000 7f75de7ef980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:24.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.758+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:24.760 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.762+0000 7f75de7ef980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:24.760 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.762+0000 7f75de7ef980 20 searching for the correct realm 2024-03-04T18:03:24.765 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.766+0000 7f75dbb06640 0 -- 172.21.15.137:0/4211769964 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f75cc031070 msgr2=0x7f75cc046750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:24.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.978+0000 7f75dbb06640 0 -- 172.21.15.137:0/4211769964 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x55dec0a3a2e0 msgr2=0x55dec0a3c6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:24.993 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:24.993 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:24.993 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:24.994 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:24.994 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:24.994 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:24.994 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:24.994 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:24.994 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:24.995 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:24.995 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:24.995 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:24.995 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 zone default found 2024-03-04T18:03:24.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 4 Realm: () 2024-03-04T18:03:24.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:24.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:24.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:24.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 zonegroup default 2024-03-04T18:03:24.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.994+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:24.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.998+0000 7f75de7ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:24.997 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.998+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:24.997 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.998+0000 7f75de7ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:24.997 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.998+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:24.998 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.998+0000 7f75de7ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:24.998 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.998+0000 7f75de7ef980 20 started sync module instance, tier type = 2024-03-04T18:03:24.998 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:24.998+0000 7f75de7ef980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:25.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.022+0000 7f75de7ef980 20 add_watcher() i=0 2024-03-04T18:03:25.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.022+0000 7f75de7ef980 20 add_watcher() i=1 2024-03-04T18:03:25.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.022+0000 7f75de7ef980 20 add_watcher() i=2 2024-03-04T18:03:25.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.022+0000 7f75de7ef980 20 add_watcher() i=3 2024-03-04T18:03:25.022 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.022+0000 7f75de7ef980 20 add_watcher() i=4 2024-03-04T18:03:25.022 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.022+0000 7f75de7ef980 20 add_watcher() i=5 2024-03-04T18:03:25.022 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.022+0000 7f75de7ef980 20 add_watcher() i=6 2024-03-04T18:03:25.024 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.022+0000 7f75de7ef980 20 add_watcher() i=7 2024-03-04T18:03:25.024 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.022+0000 7f75de7ef980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:25.027 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.026+0000 7f7492fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:25.028 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.030+0000 7f75de7ef980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:25.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.030+0000 7f75de7ef980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:25.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.030+0000 7f75de7ef980 5 note: GC not initialized 2024-03-04T18:03:25.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.030+0000 7f7491fdb640 20 reqs_thread_entry: start 2024-03-04T18:03:25.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.030+0000 7f75de7ef980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:25.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.030+0000 7f75de7ef980 20 Filter name: none 2024-03-04T18:03:25.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.030+0000 7f74037fe640 20 reqs_thread_entry: start 2024-03-04T18:03:25.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.030+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.031 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.030+0000 7f75de7ef980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:25.031 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.030+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.034+0000 7f75de7ef980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:25.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.034+0000 7f75de7ef980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:25.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.034+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.034+0000 7f75de7ef980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:25.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.034+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.034+0000 7f75de7ef980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:25.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.034+0000 7f75de7ef980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:25.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.034+0000 7f75de7ef980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.034+0000 7f75de7ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.038+0000 7f7492fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:25.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.042+0000 7f75de7ef980 20 remove_watcher() i=0 2024-03-04T18:03:25.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.042+0000 7f75de7ef980 2 removed watcher, disabling cache 2024-03-04T18:03:25.043 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.042+0000 7f75de7ef980 20 remove_watcher() i=1 2024-03-04T18:03:25.045 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.046+0000 7f75de7ef980 20 remove_watcher() i=2 2024-03-04T18:03:25.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.050+0000 7f75de7ef980 20 remove_watcher() i=3 2024-03-04T18:03:25.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.050+0000 7f75de7ef980 20 remove_watcher() i=4 2024-03-04T18:03:25.052 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.054+0000 7f75de7ef980 20 remove_watcher() i=5 2024-03-04T18:03:25.054 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.054+0000 7f75de7ef980 20 remove_watcher() i=6 2024-03-04T18:03:25.057 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.058+0000 7f75de7ef980 20 remove_watcher() i=7 2024-03-04T18:03:25.072 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:25.072 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:03:25.072 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-set-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:03:25.072 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:25.072 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout 2024-03-04T18:03:25.103 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:25.103 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:25.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.146+0000 7fcd6a4d8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.146+0000 7fcd6a4d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.146+0000 7fcd6a4d8980 20 realm 2024-03-04T18:03:25.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.146+0000 7fcd6a4d8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.146+0000 7fcd6a4d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.146+0000 7fcd6a4d8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:25.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.146+0000 7fcd6a4d8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.146+0000 7fcd6a4d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.146+0000 7fcd6a4d8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.150+0000 7fcd6a4d8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:25.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.150+0000 7fcd6a4d8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.150+0000 7fcd6a4d8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:25.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.150+0000 7fcd6a4d8980 20 searching for the correct realm 2024-03-04T18:03:25.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.154+0000 7fcd687f1640 0 -- 172.21.15.137:0/2095404180 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fcd6005a0e0 msgr2=0x7fcd600574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:25.370 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.370+0000 7fcd687f1640 0 -- 172.21.15.137:0/2095404180 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55bbfd935aa0 msgr2=0x55bbfd937f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:25.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.378+0000 7fcd6a4d8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:25.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.378+0000 7fcd6a4d8980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:25.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.378+0000 7fcd6a4d8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:25.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.378+0000 7fcd6a4d8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:25.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.378+0000 7fcd6a4d8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:25.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.378+0000 7fcd6a4d8980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:25.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.378+0000 7fcd6a4d8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.378+0000 7fcd6a4d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.378+0000 7fcd6a4d8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:25.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:25.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 20 zone default found 2024-03-04T18:03:25.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 4 Realm: () 2024-03-04T18:03:25.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:25.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:25.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:25.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 20 zonegroup default 2024-03-04T18:03:25.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.382 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 20 started sync module instance, tier type = 2024-03-04T18:03:25.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.382+0000 7fcd6a4d8980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:25.406 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.406+0000 7fcd6a4d8980 20 add_watcher() i=0 2024-03-04T18:03:25.407 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.406+0000 7fcd6a4d8980 20 add_watcher() i=1 2024-03-04T18:03:25.407 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.406+0000 7fcd6a4d8980 20 add_watcher() i=2 2024-03-04T18:03:25.407 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.406+0000 7fcd6a4d8980 20 add_watcher() i=3 2024-03-04T18:03:25.407 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.406+0000 7fcd6a4d8980 20 add_watcher() i=4 2024-03-04T18:03:25.407 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.406+0000 7fcd6a4d8980 20 add_watcher() i=5 2024-03-04T18:03:25.407 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.406+0000 7fcd6a4d8980 20 add_watcher() i=6 2024-03-04T18:03:25.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.410+0000 7fcd6a4d8980 20 add_watcher() i=7 2024-03-04T18:03:25.409 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.410+0000 7fcd6a4d8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:25.414 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.414+0000 7fcc1dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:25.415 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.414+0000 7fcd6a4d8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:25.415 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.414+0000 7fcd6a4d8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:25.415 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.414+0000 7fcd6a4d8980 5 note: GC not initialized 2024-03-04T18:03:25.415 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.414+0000 7fcc1cfe1640 20 reqs_thread_entry: start 2024-03-04T18:03:25.416 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.414+0000 7fcd6a4d8980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:25.416 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.414+0000 7fcd6a4d8980 20 Filter name: none 2024-03-04T18:03:25.416 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.418+0000 7fcb967fc640 20 reqs_thread_entry: start 2024-03-04T18:03:25.416 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.418+0000 7fcd6a4d8980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-03-04T18:03:25.416 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.418+0000 7fcd6a4d8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.417 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.418+0000 7fcd6a4d8980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:25.417 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.418+0000 7fcd6a4d8980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-03-04T18:03:25.417 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.418+0000 7fcd6a4d8980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-03-04T18:03:25.417 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.418+0000 7fcd6a4d8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 : miss 2024-03-04T18:03:25.417 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.418+0000 7fcd6a4d8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.418 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.418+0000 7fcd6a4d8980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:25.418 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.418+0000 7fcd6a4d8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 info.flags=0x17 2024-03-04T18:03:25.418 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.418+0000 7fcd6a4d8980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 to cache LRU end 2024-03-04T18:03:25.418 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.418+0000 7fcd6a4d8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:25.418 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.418+0000 7fcd6a4d8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:25.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.418+0000 7fcd6a4d8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 2024-03-04T18:03:25.423 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.422+0000 7fcd6a4d8980 -1 Injecting error=-5 at location=set_target_layout 2024-03-04T18:03:25.423 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.422+0000 7fcd6a4d8980 0 ERROR: init_target_layout failed to write target index layout to bucket info: (5) Input/output error 2024-03-04T18:03:25.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.430+0000 7fcc1dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:25.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.434+0000 7fcd6a4d8980 20 remove_watcher() i=0 2024-03-04T18:03:25.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.434+0000 7fcd6a4d8980 2 removed watcher, disabling cache 2024-03-04T18:03:25.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.434+0000 7fcd6a4d8980 20 remove_watcher() i=1 2024-03-04T18:03:25.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.438+0000 7fcd6a4d8980 20 remove_watcher() i=2 2024-03-04T18:03:25.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.442+0000 7fcd6a4d8980 20 remove_watcher() i=3 2024-03-04T18:03:25.441 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.442+0000 7fcd6a4d8980 20 remove_watcher() i=4 2024-03-04T18:03:25.444 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.446+0000 7fcd6a4d8980 20 remove_watcher() i=5 2024-03-04T18:03:25.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.446+0000 7fcd6a4d8980 20 remove_watcher() i=6 2024-03-04T18:03:25.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.450+0000 7fcd6a4d8980 20 remove_watcher() i=7 2024-03-04T18:03:25.465 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-03-04T18:03:25.496 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:25.496 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:25.539 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.538+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.542+0000 7f39e2176980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.542+0000 7f39e2176980 20 realm 2024-03-04T18:03:25.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.542+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.542+0000 7f39e2176980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.542+0000 7f39e2176980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:25.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.542+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.542+0000 7f39e2176980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.542+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.544 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.546+0000 7f39e2176980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:25.544 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.546+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.546+0000 7f39e2176980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:25.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.546+0000 7f39e2176980 20 searching for the correct realm 2024-03-04T18:03:25.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.550+0000 7f39df48d640 0 -- 172.21.15.137:0/1742489250 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f39d0008040 msgr2=0x7f39d0092040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:25.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.558+0000 7f39df48d640 0 -- 172.21.15.137:0/1742489250 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x55c5d89634a0 msgr2=0x55c5d8965890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:25.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:25.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:25.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:25.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:25.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:25.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:25.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:25.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:25.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 zone default found 2024-03-04T18:03:25.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 4 Realm: () 2024-03-04T18:03:25.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:25.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:25.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:25.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 zonegroup default 2024-03-04T18:03:25.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 started sync module instance, tier type = 2024-03-04T18:03:25.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.566+0000 7f39e2176980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:25.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.586+0000 7f39e2176980 20 add_watcher() i=0 2024-03-04T18:03:25.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.586+0000 7f39e2176980 20 add_watcher() i=1 2024-03-04T18:03:25.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.586+0000 7f39e2176980 20 add_watcher() i=2 2024-03-04T18:03:25.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.586+0000 7f39e2176980 20 add_watcher() i=3 2024-03-04T18:03:25.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.586+0000 7f39e2176980 20 add_watcher() i=4 2024-03-04T18:03:25.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.586+0000 7f39e2176980 20 add_watcher() i=5 2024-03-04T18:03:25.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.586+0000 7f39e2176980 20 add_watcher() i=6 2024-03-04T18:03:25.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.590+0000 7f39e2176980 20 add_watcher() i=7 2024-03-04T18:03:25.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.590+0000 7f39e2176980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:25.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.594+0000 7f3895fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:25.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.594+0000 7f39e2176980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:25.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.594+0000 7f39e2176980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:25.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.594+0000 7f39e2176980 5 note: GC not initialized 2024-03-04T18:03:25.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.594+0000 7f3894fe1640 20 reqs_thread_entry: start 2024-03-04T18:03:25.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.594+0000 7f39e2176980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:25.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.594+0000 7f39e2176980 20 Filter name: none 2024-03-04T18:03:25.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.594+0000 7f380e7fc640 20 reqs_thread_entry: start 2024-03-04T18:03:25.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.594+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.594+0000 7f39e2176980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:25.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.594+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.594+0000 7f39e2176980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:25.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.594+0000 7f39e2176980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:25.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.594+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.598+0000 7f39e2176980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:25.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.598+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.598+0000 7f39e2176980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:25.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.598+0000 7f39e2176980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:25.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.598+0000 7f39e2176980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.598+0000 7f39e2176980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.598+0000 7f3895fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:25.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.602+0000 7f39e2176980 20 remove_watcher() i=0 2024-03-04T18:03:25.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.602+0000 7f39e2176980 2 removed watcher, disabling cache 2024-03-04T18:03:25.604 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.606+0000 7f39e2176980 20 remove_watcher() i=1 2024-03-04T18:03:25.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.606+0000 7f39e2176980 20 remove_watcher() i=2 2024-03-04T18:03:25.609 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.610+0000 7f39e2176980 20 remove_watcher() i=3 2024-03-04T18:03:25.611 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.610+0000 7f39e2176980 20 remove_watcher() i=4 2024-03-04T18:03:25.613 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.614+0000 7f39e2176980 20 remove_watcher() i=5 2024-03-04T18:03:25.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.614+0000 7f39e2176980 20 remove_watcher() i=6 2024-03-04T18:03:25.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.618+0000 7f39e2176980 20 remove_watcher() i=7 2024-03-04T18:03:25.633 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:25.633 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:03:25.633 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-set-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:03:25.633 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:25.634 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:25.635 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:25.667 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:25.667 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:25.725 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:03:25.760 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:25.760 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:25.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.802+0000 7efe10c98980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.806+0000 7efe10c98980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.806+0000 7efe10c98980 20 realm 2024-03-04T18:03:25.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.806+0000 7efe10c98980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.806+0000 7efe10c98980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.806+0000 7efe10c98980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:25.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.806+0000 7efe10c98980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.806+0000 7efe10c98980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:25.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.806+0000 7efe10c98980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.806+0000 7efe10c98980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:25.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.806+0000 7efe10c98980 20 rados->read ofs=0 len=0 2024-03-04T18:03:25.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.806+0000 7efe10c98980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:25.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.806+0000 7efe10c98980 20 searching for the correct realm 2024-03-04T18:03:25.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:25.810+0000 7efe0efb1640 0 -- 172.21.15.137:0/799453860 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7efe080c2940 msgr2=0x7efe080c13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:26.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.026+0000 7efe0efb1640 0 -- 172.21.15.137:0/799453860 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x5607dad870d0 msgr2=0x5607dad89590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:26.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.034+0000 7efe10c98980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:26.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.034+0000 7efe10c98980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:26.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.034+0000 7efe10c98980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:26.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.034+0000 7efe10c98980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:26.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.034+0000 7efe10c98980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:26.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.034+0000 7efe10c98980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:26.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.034+0000 7efe10c98980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.034+0000 7efe10c98980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:26.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.034+0000 7efe10c98980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:26.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:26.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 20 zone default found 2024-03-04T18:03:26.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 4 Realm: () 2024-03-04T18:03:26.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:26.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:26.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:26.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 20 zonegroup default 2024-03-04T18:03:26.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:26.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:26.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:26.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 20 started sync module instance, tier type = 2024-03-04T18:03:26.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.038+0000 7efe10c98980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:26.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.066+0000 7efe10c98980 20 add_watcher() i=0 2024-03-04T18:03:26.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.066+0000 7efe10c98980 20 add_watcher() i=1 2024-03-04T18:03:26.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.066+0000 7efe10c98980 20 add_watcher() i=2 2024-03-04T18:03:26.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.066+0000 7efe10c98980 20 add_watcher() i=3 2024-03-04T18:03:26.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.066+0000 7efe10c98980 20 add_watcher() i=4 2024-03-04T18:03:26.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.066+0000 7efe10c98980 20 add_watcher() i=5 2024-03-04T18:03:26.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.066+0000 7efe10c98980 20 add_watcher() i=6 2024-03-04T18:03:26.067 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.066+0000 7efe10c98980 20 add_watcher() i=7 2024-03-04T18:03:26.067 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.066+0000 7efe10c98980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:26.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.070+0000 7efcc4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:26.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:26.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:26.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 5 note: GC not initialized 2024-03-04T18:03:26.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efc3f7fe640 20 reqs_thread_entry: start 2024-03-04T18:03:26.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:26.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 20 Filter name: none 2024-03-04T18:03:26.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efc3d7fa640 20 reqs_thread_entry: start 2024-03-04T18:03:26.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-03-04T18:03:26.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:26.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-03-04T18:03:26.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-03-04T18:03:26.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 : miss 2024-03-04T18:03:26.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:26.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 info.flags=0x17 2024-03-04T18:03:26.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 to cache LRU end 2024-03-04T18:03:26.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:26.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:26.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.074+0000 7efe10c98980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 2024-03-04T18:03:26.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.082+0000 7efe10c98980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 info.flags=0x17 2024-03-04T18:03:26.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.082+0000 7efe10c98980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 to cache LRU end 2024-03-04T18:03:26.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.082+0000 7efe10c98980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:26.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.082+0000 7efe10c98980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20, ofs0, ns] 2024-03-04T18:03:26.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.082+0000 7efcf1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627808 cookie 94591736406272 notifier 6180 bl.length()=959 2024-03-04T18:03:26.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.082+0000 7efcf1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 info.flags=0x17 2024-03-04T18:03:26.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.082+0000 7efcf1ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 to cache LRU end 2024-03-04T18:03:26.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.082+0000 7efcf1ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:26.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.086+0000 7efe10c98980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20]): orig_sources= new_sources= 2024-03-04T18:03:26.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.086+0000 7efe10c98980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20]): potential sources added=[] removed=[] 2024-03-04T18:03:26.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.086+0000 7efe10c98980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20]): orig_dests= new_dests= 2024-03-04T18:03:26.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.086+0000 7efe10c98980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20]): potential dests added=[] removed=[] 2024-03-04T18:03:26.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.090+0000 7efe10c98980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20.1.0 2024-03-04T18:03:26.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.090+0000 7efe10c98980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20.1.0 2024-03-04T18:03:26.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.090+0000 7efe10c98980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20.1.1 2024-03-04T18:03:26.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.090+0000 7efe10c98980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20.1.1 2024-03-04T18:03:26.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.090+0000 7efe10c98980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20.0 2024-03-04T18:03:26.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.094+0000 7efe10c98980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 : hit (requested=0x17, cached=0x17) 2024-03-04T18:03:26.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.094+0000 7efe10c98980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 2024-03-04T18:03:26.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.098+0000 7efe10c98980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 info.flags=0x17 2024-03-04T18:03:26.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.098+0000 7efe10c98980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 to cache LRU end 2024-03-04T18:03:26.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.098+0000 7efe10c98980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:26.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.098+0000 7efe10c98980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20, ofs0, ns] 2024-03-04T18:03:26.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.098+0000 7efcf17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627809 cookie 94591736406272 notifier 6180 bl.length()=927 2024-03-04T18:03:26.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.098+0000 7efcf17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 info.flags=0x17 2024-03-04T18:03:26.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.098+0000 7efcf17fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 to cache LRU end 2024-03-04T18:03:26.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.098+0000 7efcf17fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:26.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.098+0000 7efe10c98980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20]): orig_sources= new_sources= 2024-03-04T18:03:26.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.098+0000 7efe10c98980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20]): potential sources added=[] removed=[] 2024-03-04T18:03:26.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.098+0000 7efe10c98980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20]): orig_dests= new_dests= 2024-03-04T18:03:26.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.098+0000 7efe10c98980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20]): potential dests added=[] removed=[] 2024-03-04T18:03:26.102 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.102+0000 7efe10c98980 1 execute INFO: reshard of bucket "error-at-set-target-layout" completed successfully 2024-03-04T18:03:26.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.106+0000 7efcc4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:26.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.110+0000 7efe10c98980 20 remove_watcher() i=0 2024-03-04T18:03:26.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.110+0000 7efe10c98980 2 removed watcher, disabling cache 2024-03-04T18:03:26.113 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.114+0000 7efe10c98980 20 remove_watcher() i=1 2024-03-04T18:03:26.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.114+0000 7efe10c98980 20 remove_watcher() i=2 2024-03-04T18:03:26.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.118+0000 7efe10c98980 20 remove_watcher() i=3 2024-03-04T18:03:26.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.118+0000 7efe10c98980 20 remove_watcher() i=4 2024-03-04T18:03:26.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.122+0000 7efe10c98980 20 remove_watcher() i=5 2024-03-04T18:03:26.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.126+0000 7efe10c98980 20 remove_watcher() i=6 2024-03-04T18:03:26.128 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.130+0000 7efe10c98980 20 remove_watcher() i=7 2024-03-04T18:03:26.145 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-03-04T18:03:26.176 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:26.177 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:26.219 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.218+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.222 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.222+0000 7f03823a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:26.222 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.222+0000 7f03823a9980 20 realm 2024-03-04T18:03:26.222 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.222+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.222+0000 7f03823a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:26.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.222+0000 7f03823a9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:26.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.222+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.222+0000 7f03823a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:26.223 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.222+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.226+0000 7f03823a9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:26.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.226+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.225 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.226+0000 7f03823a9980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:26.225 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.226+0000 7f03823a9980 20 searching for the correct realm 2024-03-04T18:03:26.230 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.230+0000 7f03806c2640 0 -- 172.21.15.137:0/666053983 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f03780f6230 msgr2=0x7f03780f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:26.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.446+0000 7f03806c2640 0 -- 172.21.15.137:0/666053983 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x560845051570 msgr2=0x560845053a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:26.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.454+0000 7f03823a9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:26.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.454+0000 7f03823a9980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:26.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.454+0000 7f03823a9980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:26.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.454+0000 7f03823a9980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:26.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.454+0000 7f03823a9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:26.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.454+0000 7f03823a9980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:26.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.454+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.454+0000 7f03823a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:26.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.454+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:26.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:26.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 20 zone default found 2024-03-04T18:03:26.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 4 Realm: () 2024-03-04T18:03:26.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:26.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:26.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:26.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 20 zonegroup default 2024-03-04T18:03:26.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:26.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:26.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.459 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:26.459 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 20 started sync module instance, tier type = 2024-03-04T18:03:26.459 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.458+0000 7f03823a9980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:26.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.482+0000 7f03823a9980 20 add_watcher() i=0 2024-03-04T18:03:26.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.482+0000 7f03823a9980 20 add_watcher() i=1 2024-03-04T18:03:26.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.482+0000 7f03823a9980 20 add_watcher() i=2 2024-03-04T18:03:26.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.482+0000 7f03823a9980 20 add_watcher() i=3 2024-03-04T18:03:26.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.482+0000 7f03823a9980 20 add_watcher() i=4 2024-03-04T18:03:26.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.482+0000 7f03823a9980 20 add_watcher() i=5 2024-03-04T18:03:26.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.482+0000 7f03823a9980 20 add_watcher() i=6 2024-03-04T18:03:26.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.486+0000 7f03823a9980 20 add_watcher() i=7 2024-03-04T18:03:26.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.486+0000 7f03823a9980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:26.490 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.490+0000 7f0235fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:26.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.490+0000 7f03823a9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:26.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.490+0000 7f03823a9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:26.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.490+0000 7f03823a9980 5 note: GC not initialized 2024-03-04T18:03:26.491 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.490+0000 7f0234fe1640 20 reqs_thread_entry: start 2024-03-04T18:03:26.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.490+0000 7f03823a9980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:26.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.490+0000 7f03823a9980 20 Filter name: none 2024-03-04T18:03:26.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.494+0000 7f01ae7fc640 20 reqs_thread_entry: start 2024-03-04T18:03:26.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.494+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.493 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.494+0000 7f03823a9980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:26.494 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.494+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.494+0000 7f03823a9980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:26.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.494+0000 7f03823a9980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:26.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.494+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.498+0000 7f03823a9980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:26.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.498+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.498+0000 7f03823a9980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:26.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.498+0000 7f03823a9980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:26.497 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.498+0000 7f03823a9980 20 rados->read ofs=0 len=0 2024-03-04T18:03:26.497 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.498+0000 7f03823a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:26.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.498+0000 7f0235fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:26.503 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.502+0000 7f03823a9980 20 remove_watcher() i=0 2024-03-04T18:03:26.503 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.502+0000 7f03823a9980 2 removed watcher, disabling cache 2024-03-04T18:03:26.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.506+0000 7f03823a9980 20 remove_watcher() i=1 2024-03-04T18:03:26.507 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.506+0000 7f03823a9980 20 remove_watcher() i=2 2024-03-04T18:03:26.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.510+0000 7f03823a9980 20 remove_watcher() i=3 2024-03-04T18:03:26.512 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.514+0000 7f03823a9980 20 remove_watcher() i=4 2024-03-04T18:03:26.514 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.514+0000 7f03823a9980 20 remove_watcher() i=5 2024-03-04T18:03:26.517 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.518+0000 7f03823a9980 20 remove_watcher() i=6 2024-03-04T18:03:26.520 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:26.518+0000 7f03823a9980 20 remove_watcher() i=7 2024-03-04T18:03:26.535 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:26.535 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:03:26.535 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-set-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.20 num_objects: 19 size_kb: 1 num_shards: 2 2024-03-04T18:03:26.535 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:26.537 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:26.537 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:26.554 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:26.554 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:26.937 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:26.937 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:27.124 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: reshard bucket with ECANCELED injected at set_target_layout 2024-03-04T18:03:27.124 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:27.126 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:27.126 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:27.236 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:27.236 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:27.315 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:27.315 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:27.399 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:27.400 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:27.479 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:27.480 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:27.564 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:27.564 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:27.648 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:27.648 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:27.734 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:27.735 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:27.821 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:27.821 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:27.903 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:27.903 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:27.984 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:27.984 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:28.067 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:28.068 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:28.153 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:28.153 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:28.238 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:28.238 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:28.307 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:28.307 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:28.393 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:28.394 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:28.475 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:28.475 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:28.544 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:28.544 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:28.628 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:28.628 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:28.710 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:28.710 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:28.794 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:28.794 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:28.875 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:28.875 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:28.960 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-03-04T18:03:28.993 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:28.993 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:29.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.038+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.038+0000 7fda21c30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.038+0000 7fda21c30980 20 realm 2024-03-04T18:03:29.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.038+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.038+0000 7fda21c30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.038+0000 7fda21c30980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:29.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.038+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.038+0000 7fda21c30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.038+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.042+0000 7fda21c30980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:29.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.042+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.042+0000 7fda21c30980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:29.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.042+0000 7fda21c30980 20 searching for the correct realm 2024-03-04T18:03:29.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.046+0000 7fda1ff49640 0 -- 172.21.15.137:0/3221264006 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fda1801f930 msgr2=0x7fda1801e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:29.261 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.262+0000 7fda1ff49640 0 -- 172.21.15.137:0/3221264006 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x5638af88cad0 msgr2=0x5638af88ef90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:29.271 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.270+0000 7fda21c30980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:29.271 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.270+0000 7fda21c30980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:29.271 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.270+0000 7fda21c30980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:29.271 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.270+0000 7fda21c30980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:29.272 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.270+0000 7fda21c30980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:29.272 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.270+0000 7fda21c30980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:29.272 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.270+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.272 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.272 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.273 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:29.273 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.273 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:29.273 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 20 zone default found 2024-03-04T18:03:29.273 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 4 Realm: () 2024-03-04T18:03:29.273 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:29.274 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:29.274 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:29.274 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 20 zonegroup default 2024-03-04T18:03:29.274 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.274 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.274 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.275 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.275 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.274+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.276 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.278+0000 7fda21c30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.276 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.278+0000 7fda21c30980 20 started sync module instance, tier type = 2024-03-04T18:03:29.276 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.278+0000 7fda21c30980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:29.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.298+0000 7fda21c30980 20 add_watcher() i=0 2024-03-04T18:03:29.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.298+0000 7fda21c30980 20 add_watcher() i=1 2024-03-04T18:03:29.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.298+0000 7fda21c30980 20 add_watcher() i=2 2024-03-04T18:03:29.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.298+0000 7fda21c30980 20 add_watcher() i=3 2024-03-04T18:03:29.300 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.298+0000 7fda21c30980 20 add_watcher() i=4 2024-03-04T18:03:29.300 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.298+0000 7fda21c30980 20 add_watcher() i=5 2024-03-04T18:03:29.300 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.298+0000 7fda21c30980 20 add_watcher() i=6 2024-03-04T18:03:29.302 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.302+0000 7fda21c30980 20 add_watcher() i=7 2024-03-04T18:03:29.302 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.302+0000 7fda21c30980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:29.306 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.306+0000 7fd8d5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:29.307 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.306+0000 7fda21c30980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:29.307 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.306+0000 7fda21c30980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:29.307 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.306+0000 7fda21c30980 5 note: GC not initialized 2024-03-04T18:03:29.308 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.306+0000 7fd8d4fe1640 20 reqs_thread_entry: start 2024-03-04T18:03:29.308 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.306+0000 7fda21c30980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:29.308 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.310+0000 7fda21c30980 20 Filter name: none 2024-03-04T18:03:29.308 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.310+0000 7fd84e7fc640 20 reqs_thread_entry: start 2024-03-04T18:03:29.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.310+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.310+0000 7fda21c30980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:29.310 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.310+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.311 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.310+0000 7fda21c30980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:29.311 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.310+0000 7fda21c30980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:29.311 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.310+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.314+0000 7fda21c30980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:29.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.314+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.314+0000 7fda21c30980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:29.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.314+0000 7fda21c30980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:29.314 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.314+0000 7fda21c30980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.314 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.314+0000 7fda21c30980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.318+0000 7fd8d5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:29.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.322+0000 7fda21c30980 20 remove_watcher() i=0 2024-03-04T18:03:29.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.322+0000 7fda21c30980 2 removed watcher, disabling cache 2024-03-04T18:03:29.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.330+0000 7fda21c30980 20 remove_watcher() i=1 2024-03-04T18:03:29.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.330+0000 7fda21c30980 20 remove_watcher() i=2 2024-03-04T18:03:29.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.334+0000 7fda21c30980 20 remove_watcher() i=3 2024-03-04T18:03:29.337 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.338+0000 7fda21c30980 20 remove_watcher() i=4 2024-03-04T18:03:29.339 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.338+0000 7fda21c30980 20 remove_watcher() i=5 2024-03-04T18:03:29.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.342+0000 7fda21c30980 20 remove_watcher() i=6 2024-03-04T18:03:29.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.346+0000 7fda21c30980 20 remove_watcher() i=7 2024-03-04T18:03:29.362 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:29.363 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:03:29.363 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-set-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:03:29.363 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:29.363 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout --inject-error-code 125 2024-03-04T18:03:29.401 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:29.401 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:29.444 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.446+0000 7fa31d782980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.446+0000 7fa31d782980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.446+0000 7fa31d782980 20 realm 2024-03-04T18:03:29.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.446+0000 7fa31d782980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.446+0000 7fa31d782980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.446+0000 7fa31d782980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:29.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.446+0000 7fa31d782980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.446+0000 7fa31d782980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.446+0000 7fa31d782980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.448 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.446+0000 7fa31d782980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:29.448 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.446+0000 7fa31d782980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.448 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.450+0000 7fa31d782980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:29.448 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.450+0000 7fa31d782980 20 searching for the correct realm 2024-03-04T18:03:29.451 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.450+0000 7fa31ba9b640 0 -- 172.21.15.137:0/593957571 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x56112d603af0 msgr2=0x56112d605f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:29.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.458+0000 7fa31b29a640 0 -- 172.21.15.137:0/593957571 >> [v2:172.21.15.173:6805/879938785,v1:172.21.15.173:6808/879938785] conn(0x7fa30c0b1510 msgr2=0x7fa30c0f1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:03:29.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.678+0000 7fa31d782980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:29.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.678+0000 7fa31d782980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:29.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.678+0000 7fa31d782980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:29.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.678+0000 7fa31d782980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:29.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.678+0000 7fa31d782980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:29.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.678+0000 7fa31d782980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:29.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.678+0000 7fa31d782980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.678+0000 7fa31d782980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.678+0000 7fa31d782980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:29.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:29.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 20 zone default found 2024-03-04T18:03:29.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 4 Realm: () 2024-03-04T18:03:29.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:29.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:29.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:29.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 20 zonegroup default 2024-03-04T18:03:29.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 20 started sync module instance, tier type = 2024-03-04T18:03:29.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.682+0000 7fa31d782980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:29.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.706+0000 7fa31d782980 20 add_watcher() i=0 2024-03-04T18:03:29.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.706+0000 7fa31d782980 20 add_watcher() i=1 2024-03-04T18:03:29.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.706+0000 7fa31d782980 20 add_watcher() i=2 2024-03-04T18:03:29.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.706+0000 7fa31d782980 20 add_watcher() i=3 2024-03-04T18:03:29.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.706+0000 7fa31d782980 20 add_watcher() i=4 2024-03-04T18:03:29.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.706+0000 7fa31d782980 20 add_watcher() i=5 2024-03-04T18:03:29.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.706+0000 7fa31d782980 20 add_watcher() i=6 2024-03-04T18:03:29.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.710+0000 7fa31d782980 20 add_watcher() i=7 2024-03-04T18:03:29.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.710+0000 7fa31d782980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:29.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.714+0000 7fa1d27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:29.714 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.714+0000 7fa31d782980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:29.714 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.714+0000 7fa31d782980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:29.714 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.714+0000 7fa31d782980 5 note: GC not initialized 2024-03-04T18:03:29.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.714+0000 7fa1d17e2640 20 reqs_thread_entry: start 2024-03-04T18:03:29.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.714+0000 7fa31d782980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:29.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.714+0000 7fa31d782980 20 Filter name: none 2024-03-04T18:03:29.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.714+0000 7fa1ca7fc640 20 reqs_thread_entry: start 2024-03-04T18:03:29.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.714+0000 7fa31d782980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-03-04T18:03:29.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.714+0000 7fa31d782980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.718+0000 7fa31d782980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:29.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.718+0000 7fa31d782980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-03-04T18:03:29.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.718+0000 7fa31d782980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-03-04T18:03:29.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.718+0000 7fa31d782980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 : miss 2024-03-04T18:03:29.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.718+0000 7fa31d782980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.718+0000 7fa31d782980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:29.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.718+0000 7fa31d782980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 info.flags=0x17 2024-03-04T18:03:29.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.718+0000 7fa31d782980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 to cache LRU end 2024-03-04T18:03:29.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.718+0000 7fa31d782980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:29.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.718+0000 7fa31d782980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:29.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.718+0000 7fa31d782980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 2024-03-04T18:03:29.723 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.722+0000 7fa31d782980 -1 Injecting error=-125 at location=set_target_layout 2024-03-04T18:03:29.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.726+0000 7fa31d782980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 info.flags=0x17 2024-03-04T18:03:29.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.726+0000 7fa31d782980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 to cache LRU end 2024-03-04T18:03:29.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.726+0000 7fa31d782980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:29.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.726+0000 7fa31d782980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21, ofs0, ns] 2024-03-04T18:03:29.728 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.726+0000 7fa1feffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627812 cookie 94631775794736 notifier 6237 bl.length()=959 2024-03-04T18:03:29.728 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.726+0000 7fa1feffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 info.flags=0x17 2024-03-04T18:03:29.728 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.726+0000 7fa1feffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 to cache LRU end 2024-03-04T18:03:29.728 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.726+0000 7fa1feffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:29.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.734+0000 7fa31d782980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21]): orig_sources= new_sources= 2024-03-04T18:03:29.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.734+0000 7fa31d782980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21]): potential sources added=[] removed=[] 2024-03-04T18:03:29.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.734+0000 7fa31d782980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21]): orig_dests= new_dests= 2024-03-04T18:03:29.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.734+0000 7fa31d782980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21]): potential dests added=[] removed=[] 2024-03-04T18:03:29.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.734+0000 7fa31d782980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21.1.0 2024-03-04T18:03:29.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.734+0000 7fa31d782980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21.1.0 2024-03-04T18:03:29.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.734+0000 7fa31d782980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21.1.1 2024-03-04T18:03:29.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.734+0000 7fa31d782980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21.1.1 2024-03-04T18:03:29.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.734+0000 7fa31d782980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21.0 2024-03-04T18:03:29.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.738+0000 7fa31d782980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 : hit (requested=0x17, cached=0x17) 2024-03-04T18:03:29.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.738+0000 7fa31d782980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 2024-03-04T18:03:29.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.742+0000 7fa31d782980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 info.flags=0x17 2024-03-04T18:03:29.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.742+0000 7fa31d782980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 to cache LRU end 2024-03-04T18:03:29.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.742+0000 7fa31d782980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:29.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.742+0000 7fa31d782980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21, ofs0, ns] 2024-03-04T18:03:29.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.742+0000 7fa1fe7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627813 cookie 94631775794736 notifier 6237 bl.length()=927 2024-03-04T18:03:29.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.742+0000 7fa1fe7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 info.flags=0x17 2024-03-04T18:03:29.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.742+0000 7fa1fe7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 to cache LRU end 2024-03-04T18:03:29.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.742+0000 7fa1fe7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:29.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.746+0000 7fa31d782980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21]): orig_sources= new_sources= 2024-03-04T18:03:29.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.746+0000 7fa31d782980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21]): potential sources added=[] removed=[] 2024-03-04T18:03:29.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.746+0000 7fa31d782980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21]): orig_dests= new_dests= 2024-03-04T18:03:29.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.746+0000 7fa31d782980 20 handle_bi_update(): bucket=:error-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21]): potential dests added=[] removed=[] 2024-03-04T18:03:29.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.746+0000 7fa31d782980 1 execute INFO: reshard of bucket "error-at-set-target-layout" completed successfully 2024-03-04T18:03:29.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.750+0000 7fa1d27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:29.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.754+0000 7fa31d782980 20 remove_watcher() i=0 2024-03-04T18:03:29.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.754+0000 7fa31d782980 2 removed watcher, disabling cache 2024-03-04T18:03:29.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.758+0000 7fa31d782980 20 remove_watcher() i=1 2024-03-04T18:03:29.760 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.762+0000 7fa31d782980 20 remove_watcher() i=2 2024-03-04T18:03:29.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.762+0000 7fa31d782980 20 remove_watcher() i=3 2024-03-04T18:03:29.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.766+0000 7fa31d782980 20 remove_watcher() i=4 2024-03-04T18:03:29.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.770+0000 7fa31d782980 20 remove_watcher() i=5 2024-03-04T18:03:29.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.774+0000 7fa31d782980 20 remove_watcher() i=6 2024-03-04T18:03:29.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.782+0000 7fa31d782980 20 remove_watcher() i=7 2024-03-04T18:03:29.804 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-03-04T18:03:29.832 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:29.832 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:29.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.878+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.882+0000 7fe3a8bbd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.882+0000 7fe3a8bbd980 20 realm 2024-03-04T18:03:29.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.882+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.882+0000 7fe3a8bbd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.882+0000 7fe3a8bbd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:29.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.882+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.882+0000 7fe3a8bbd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:29.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.882+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.882+0000 7fe3a8bbd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:29.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.882+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:29.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.882+0000 7fe3a8bbd980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:29.884 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.882+0000 7fe3a8bbd980 20 searching for the correct realm 2024-03-04T18:03:29.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:29.890+0000 7fe3a6ed6640 0 -- 172.21.15.137:0/3269309059 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fe3a000e040 msgr2=0x7fe3a002b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:30.105 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.106+0000 7fe3a6ed6640 0 -- 172.21.15.137:0/3269309059 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x558f36bf5bb0 msgr2=0x558f36bf8070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:30.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.114+0000 7fe3a8bbd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:30.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.114+0000 7fe3a8bbd980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:30.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.114+0000 7fe3a8bbd980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:30.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.114+0000 7fe3a8bbd980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:30.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.114+0000 7fe3a8bbd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:30.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.114+0000 7fe3a8bbd980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:30.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.114+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.114+0000 7fe3a8bbd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:30.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.114+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.118+0000 7fe3a8bbd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:30.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.118+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.118+0000 7fe3a8bbd980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:30.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.118+0000 7fe3a8bbd980 20 zone default found 2024-03-04T18:03:30.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.118+0000 7fe3a8bbd980 4 Realm: () 2024-03-04T18:03:30.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.118+0000 7fe3a8bbd980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:30.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.118+0000 7fe3a8bbd980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:30.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.118+0000 7fe3a8bbd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:30.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.118+0000 7fe3a8bbd980 20 zonegroup default 2024-03-04T18:03:30.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.118+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.118+0000 7fe3a8bbd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:30.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.118+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.118+0000 7fe3a8bbd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:30.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.118+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.122+0000 7fe3a8bbd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:30.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.122+0000 7fe3a8bbd980 20 started sync module instance, tier type = 2024-03-04T18:03:30.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.122+0000 7fe3a8bbd980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:30.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.146+0000 7fe3a8bbd980 20 add_watcher() i=0 2024-03-04T18:03:30.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.146+0000 7fe3a8bbd980 20 add_watcher() i=1 2024-03-04T18:03:30.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.146+0000 7fe3a8bbd980 20 add_watcher() i=2 2024-03-04T18:03:30.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.146+0000 7fe3a8bbd980 20 add_watcher() i=3 2024-03-04T18:03:30.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.146+0000 7fe3a8bbd980 20 add_watcher() i=4 2024-03-04T18:03:30.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.146+0000 7fe3a8bbd980 20 add_watcher() i=5 2024-03-04T18:03:30.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.146+0000 7fe3a8bbd980 20 add_watcher() i=6 2024-03-04T18:03:30.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.150+0000 7fe3a8bbd980 20 add_watcher() i=7 2024-03-04T18:03:30.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.150+0000 7fe3a8bbd980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:30.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.154+0000 7fe25cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:30.153 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.154+0000 7fe3a8bbd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:30.153 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.154+0000 7fe3a8bbd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:30.153 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.154+0000 7fe3a8bbd980 5 note: GC not initialized 2024-03-04T18:03:30.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.154+0000 7fe1d77fe640 20 reqs_thread_entry: start 2024-03-04T18:03:30.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.154+0000 7fe3a8bbd980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:30.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.154+0000 7fe3a8bbd980 20 Filter name: none 2024-03-04T18:03:30.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.154+0000 7fe1d57fa640 20 reqs_thread_entry: start 2024-03-04T18:03:30.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.154+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.154+0000 7fe3a8bbd980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:30.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.154+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.158+0000 7fe3a8bbd980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:30.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.158+0000 7fe3a8bbd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:30.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.158+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.158+0000 7fe3a8bbd980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:30.158 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.158+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.159 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.158+0000 7fe3a8bbd980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:30.159 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.158+0000 7fe3a8bbd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:30.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.162+0000 7fe3a8bbd980 20 rados->read ofs=0 len=0 2024-03-04T18:03:30.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.162+0000 7fe3a8bbd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:30.162 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.162+0000 7fe25cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:30.166 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.166+0000 7fe3a8bbd980 20 remove_watcher() i=0 2024-03-04T18:03:30.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.166+0000 7fe3a8bbd980 2 removed watcher, disabling cache 2024-03-04T18:03:30.169 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.170+0000 7fe3a8bbd980 20 remove_watcher() i=1 2024-03-04T18:03:30.171 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.170+0000 7fe3a8bbd980 20 remove_watcher() i=2 2024-03-04T18:03:30.173 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.174+0000 7fe3a8bbd980 20 remove_watcher() i=3 2024-03-04T18:03:30.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.174+0000 7fe3a8bbd980 20 remove_watcher() i=4 2024-03-04T18:03:30.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.178+0000 7fe3a8bbd980 20 remove_watcher() i=5 2024-03-04T18:03:30.180 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.182+0000 7fe3a8bbd980 20 remove_watcher() i=6 2024-03-04T18:03:30.183 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:30.182+0000 7fe3a8bbd980 20 remove_watcher() i=7 2024-03-04T18:03:30.205 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:30.205 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:03:30.205 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-set-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.21 num_objects: 20 size_kb: 1 num_shards: 2 2024-03-04T18:03:30.205 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:30.207 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:30.207 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:30.228 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:30.229 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:30.663 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:30.663 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:30.837 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: reshard bucket with abort at set_target_layout 2024-03-04T18:03:30.838 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:30.841 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:30.841 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:30.955 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:30.955 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:31.036 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:31.036 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:31.121 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:31.121 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:31.220 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:31.220 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:31.302 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:31.302 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:31.386 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:31.386 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:31.471 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:31.471 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:31.555 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:31.555 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:31.639 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:31.639 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:31.725 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:31.725 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:31.810 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:31.810 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:31.892 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:31.892 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:31.973 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:31.973 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:32.056 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:32.056 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:32.140 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:32.140 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:32.224 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:32.224 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:32.309 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:32.310 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:32.391 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:32.392 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:32.477 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:32.478 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:32.563 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:32.563 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:32.644 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:32.644 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:32.727 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-03-04T18:03:32.760 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:32.761 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:32.803 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:32.802+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:32.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:32.806+0000 7faf04832980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:32.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:32.806+0000 7faf04832980 20 realm 2024-03-04T18:03:32.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:32.806+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:32.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:32.806+0000 7faf04832980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:32.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:32.806+0000 7faf04832980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:32.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:32.806+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:32.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:32.806+0000 7faf04832980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:32.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:32.806+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:32.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:32.806+0000 7faf04832980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:32.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:32.806+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:32.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:32.806+0000 7faf04832980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:32.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:32.806+0000 7faf04832980 20 searching for the correct realm 2024-03-04T18:03:32.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:32.814+0000 7faf02b4b640 0 -- 172.21.15.137:0/3929650341 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7faefc077a00 msgr2=0x7faefc05a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:33.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.030+0000 7faf02b4b640 0 -- 172.21.15.137:0/3929650341 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x561d92949ea0 msgr2=0x561d9294c360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:33.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.038+0000 7faf04832980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:33.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.038+0000 7faf04832980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:33.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.038+0000 7faf04832980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:33.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.038+0000 7faf04832980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:33.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.038+0000 7faf04832980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:33.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.038+0000 7faf04832980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:33.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.038+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.038+0000 7faf04832980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.038+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:33.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:33.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 20 zone default found 2024-03-04T18:03:33.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 4 Realm: () 2024-03-04T18:03:33.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:33.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:33.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:33.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 20 zonegroup default 2024-03-04T18:03:33.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.043 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.043 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 20 started sync module instance, tier type = 2024-03-04T18:03:33.043 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.042+0000 7faf04832980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:33.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.066+0000 7faf04832980 20 add_watcher() i=0 2024-03-04T18:03:33.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.066+0000 7faf04832980 20 add_watcher() i=1 2024-03-04T18:03:33.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.066+0000 7faf04832980 20 add_watcher() i=2 2024-03-04T18:03:33.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.066+0000 7faf04832980 20 add_watcher() i=3 2024-03-04T18:03:33.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.066+0000 7faf04832980 20 add_watcher() i=4 2024-03-04T18:03:33.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.066+0000 7faf04832980 20 add_watcher() i=5 2024-03-04T18:03:33.066 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.066+0000 7faf04832980 20 add_watcher() i=6 2024-03-04T18:03:33.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.066+0000 7faf04832980 20 add_watcher() i=7 2024-03-04T18:03:33.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.066+0000 7faf04832980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:33.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.070+0000 7fadb8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:33.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7faf04832980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:33.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7faf04832980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:33.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7faf04832980 5 note: GC not initialized 2024-03-04T18:03:33.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7fad337fe640 20 reqs_thread_entry: start 2024-03-04T18:03:33.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7faf04832980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:33.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7faf04832980 20 Filter name: none 2024-03-04T18:03:33.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7fad317fa640 20 reqs_thread_entry: start 2024-03-04T18:03:33.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7faf04832980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:33.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7faf04832980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:33.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7faf04832980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:33.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7faf04832980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:33.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.074+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.078+0000 7faf04832980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:33.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.078+0000 7faf04832980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:33.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.078+0000 7faf04832980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.078+0000 7faf04832980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.078 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.078+0000 7fadb8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:33.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.082+0000 7faf04832980 20 remove_watcher() i=0 2024-03-04T18:03:33.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.082+0000 7faf04832980 2 removed watcher, disabling cache 2024-03-04T18:03:33.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.086+0000 7faf04832980 20 remove_watcher() i=1 2024-03-04T18:03:33.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.086+0000 7faf04832980 20 remove_watcher() i=2 2024-03-04T18:03:33.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.090+0000 7faf04832980 20 remove_watcher() i=3 2024-03-04T18:03:33.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.090+0000 7faf04832980 20 remove_watcher() i=4 2024-03-04T18:03:33.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.094+0000 7faf04832980 20 remove_watcher() i=5 2024-03-04T18:03:33.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.094+0000 7faf04832980 20 remove_watcher() i=6 2024-03-04T18:03:33.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.098+0000 7faf04832980 20 remove_watcher() i=7 2024-03-04T18:03:33.112 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.112 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:03:33.112 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: abort-at-set-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:03:33.112 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.113 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at set_target_layout 2024-03-04T18:03:33.148 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:33.149 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:33.190 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.190+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.194+0000 7f6faaad8980 20 realm 2024-03-04T18:03:33.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.194+0000 7f6faaad8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:33.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:33.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:33.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.194+0000 7f6faaad8980 20 searching for the correct realm 2024-03-04T18:03:33.200 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.202+0000 7f6fa8df1640 0 -- 172.21.15.137:0/2758601501 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f6fa00c9400 msgr2=0x7f6fa00c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:03:33.415 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.414+0000 7f6fa7def640 0 -- 172.21.15.137:0/2758601501 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x560b3b9f7070 msgr2=0x560b3b9f9480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:33.428 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:33.428 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:33.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:33.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:33.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:33.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:33.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.430 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:33.430 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.431 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:33.431 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 zone default found 2024-03-04T18:03:33.431 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 4 Realm: () 2024-03-04T18:03:33.431 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:33.431 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:33.431 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:33.431 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 zonegroup default 2024-03-04T18:03:33.431 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.434+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.434+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.434+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.434+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.434+0000 7f6faaad8980 20 started sync module instance, tier type = 2024-03-04T18:03:33.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.434+0000 7f6faaad8980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:33.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=0 2024-03-04T18:03:33.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=1 2024-03-04T18:03:33.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=2 2024-03-04T18:03:33.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=3 2024-03-04T18:03:33.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=4 2024-03-04T18:03:33.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=5 2024-03-04T18:03:33.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=6 2024-03-04T18:03:33.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.458+0000 7f6faaad8980 20 add_watcher() i=7 2024-03-04T18:03:33.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.458+0000 7f6faaad8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:33.462 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.462+0000 7f6e5efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:33.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.462+0000 7f6faaad8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:33.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.462+0000 7f6faaad8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:33.463 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.462+0000 7f6faaad8980 5 note: GC not initialized 2024-03-04T18:03:33.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.462+0000 7f6e5dfdb640 20 reqs_thread_entry: start 2024-03-04T18:03:33.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.462+0000 7f6faaad8980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:33.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6faaad8980 20 Filter name: none 2024-03-04T18:03:33.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6dcf7fe640 20 reqs_thread_entry: start 2024-03-04T18:03:33.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6faaad8980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-03-04T18:03:33.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:33.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6faaad8980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-03-04T18:03:33.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6faaad8980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-03-04T18:03:33.465 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6faaad8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 : miss 2024-03-04T18:03:33.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:33.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6faaad8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 info.flags=0x17 2024-03-04T18:03:33.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6faaad8980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 to cache LRU end 2024-03-04T18:03:33.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6faaad8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:33.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6faaad8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:33.466 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.466+0000 7f6faaad8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 2024-03-04T18:03:33.469 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f6faaad8980 time 2024-03-04T18:03:33.473595+0000 2024-03-04T18:03:33.469 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:03:33.471 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:03:33.471 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f6fae3add7b] 2024-03-04T18:03:33.471 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f6fae3adf2f] 2024-03-04T18:03:33.471 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560b39462281] 2024-03-04T18:03:33.471 INFO:tasks.workunit.client.0.smithi137.stderr: 4: radosgw-admin(+0x8c3b2a) [0x560b39469b2a] 2024-03-04T18:03:33.471 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x560b3946ec93] 2024-03-04T18:03:33.471 INFO:tasks.workunit.client.0.smithi137.stderr: 6: main() 2024-03-04T18:03:33.472 INFO:tasks.workunit.client.0.smithi137.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6fad218d90] 2024-03-04T18:03:33.472 INFO:tasks.workunit.client.0.smithi137.stderr: 8: __libc_start_main() 2024-03-04T18:03:33.472 INFO:tasks.workunit.client.0.smithi137.stderr: 9: _start() 2024-03-04T18:03:33.472 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.470+0000 7f6faaad8980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f6faaad8980 time 2024-03-04T18:03:33.473595+0000 2024-03-04T18:03:33.472 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:03:33.472 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.472 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:03:33.472 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f6fae3add7b] 2024-03-04T18:03:33.472 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f6fae3adf2f] 2024-03-04T18:03:33.472 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560b39462281] 2024-03-04T18:03:33.472 INFO:tasks.workunit.client.0.smithi137.stderr: 4: radosgw-admin(+0x8c3b2a) [0x560b39469b2a] 2024-03-04T18:03:33.472 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x560b3946ec93] 2024-03-04T18:03:33.472 INFO:tasks.workunit.client.0.smithi137.stderr: 6: main() 2024-03-04T18:03:33.472 INFO:tasks.workunit.client.0.smithi137.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6fad218d90] 2024-03-04T18:03:33.473 INFO:tasks.workunit.client.0.smithi137.stderr: 8: __libc_start_main() 2024-03-04T18:03:33.473 INFO:tasks.workunit.client.0.smithi137.stderr: 9: _start() 2024-03-04T18:03:33.473 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.473 INFO:tasks.workunit.client.0.smithi137.stderr:*** Caught signal (Aborted) ** 2024-03-04T18:03:33.473 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7f6faaad8980 thread_name:radosgw-admin 2024-03-04T18:03:33.493 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:03:33.493 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f6fad231520] 2024-03-04T18:03:33.493 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:03:33.493 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:03:33.493 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f6fae3addd5] 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f6fae3adf2f] 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560b39462281] 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr: 8: radosgw-admin(+0x8c3b2a) [0x560b39469b2a] 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x560b3946ec93] 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr: 10: main() 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6fad218d90] 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr: 12: __libc_start_main() 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr: 13: _start() 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.474+0000 7f6faaad8980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7f6faaad8980 thread_name:radosgw-admin 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f6fad231520] 2024-03-04T18:03:33.494 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f6fae3addd5] 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f6fae3adf2f] 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560b39462281] 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr: 8: radosgw-admin(+0x8c3b2a) [0x560b39469b2a] 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x560b3946ec93] 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr: 10: main() 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6fad218d90] 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr: 12: __libc_start_main() 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr: 13: _start() 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr:--- begin dump of recent events --- 2024-03-04T18:03:33.495 INFO:tasks.workunit.client.0.smithi137.stderr: -499> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.496 INFO:tasks.workunit.client.0.smithi137.stderr: -498> 2024-03-04T18:03:33.170+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.496 INFO:tasks.workunit.client.0.smithi137.stderr: -497> 2024-03-04T18:03:33.170+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.496 INFO:tasks.workunit.client.0.smithi137.stderr: -496> 2024-03-04T18:03:33.170+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.496 INFO:tasks.workunit.client.0.smithi137.stderr: -495> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.496 INFO:tasks.workunit.client.0.smithi137.stderr: -494> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.496 INFO:tasks.workunit.client.0.smithi137.stderr: -493> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.496 INFO:tasks.workunit.client.0.smithi137.stderr: -492> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_done global_id 6285 payload 274 2024-03-04T18:03:33.496 INFO:tasks.workunit.client.0.smithi137.stderr: -491> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 10 monclient: _finish_hunting 0 2024-03-04T18:03:33.496 INFO:tasks.workunit.client.0.smithi137.stderr: -490> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 1 monclient: found mon.noname-a 2024-03-04T18:03:33.496 INFO:tasks.workunit.client.0.smithi137.stderr: -489> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:03:33.496 INFO:tasks.workunit.client.0.smithi137.stderr: -488> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.496 INFO:tasks.workunit.client.0.smithi137.stderr: -487> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:03:33.496 INFO:tasks.workunit.client.0.smithi137.stderr: -486> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: dump: 2024-03-04T18:03:33.496 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr: -485> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: _finish_auth 0 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr: -484> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: _check_auth_tickets 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr: -483> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr: -482> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr: -481> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr: -480> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: dump: 2024-03-04T18:03:33.497 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr: -479> 2024-03-04T18:03:33.170+0000 7f6faaad8980 5 monclient: authenticate success, global_id 6285 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr: -478> 2024-03-04T18:03:33.170+0000 7f6faaad8980 10 monclient: get_monmap_and_config success 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr: -477> 2024-03-04T18:03:33.170+0000 7f6faaad8980 4 set_mon_vals no callback set 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr: -476> 2024-03-04T18:03:33.170+0000 7f6faaad8980 10 monclient: shutdown 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr: -475> 2024-03-04T18:03:33.170+0000 7f6faaad8980 5 asok(0x560b3b71f130) unregister_commands rotate-key 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr: -474> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: build_initial_monmap 2024-03-04T18:03:33.498 INFO:tasks.workunit.client.0.smithi137.stderr: -473> 2024-03-04T18:03:33.174+0000 7f6faaad8980 1 build_initial for_mkfs: 0 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr: -472> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: monmap: 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr: -471> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding auth protocol: cephx 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr: -470> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding auth protocol: cephx 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr: -469> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding auth protocol: cephx 2024-03-04T18:03:33.499 INFO:tasks.workunit.client.0.smithi137.stderr: -468> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: secure 2024-03-04T18:03:33.500 INFO:tasks.workunit.client.0.smithi137.stderr: -467> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: crc 2024-03-04T18:03:33.500 INFO:tasks.workunit.client.0.smithi137.stderr: -466> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: secure 2024-03-04T18:03:33.500 INFO:tasks.workunit.client.0.smithi137.stderr: -465> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: crc 2024-03-04T18:03:33.500 INFO:tasks.workunit.client.0.smithi137.stderr: -464> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: secure 2024-03-04T18:03:33.500 INFO:tasks.workunit.client.0.smithi137.stderr: -463> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: crc 2024-03-04T18:03:33.500 INFO:tasks.workunit.client.0.smithi137.stderr: -462> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: crc 2024-03-04T18:03:33.500 INFO:tasks.workunit.client.0.smithi137.stderr: -461> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: secure 2024-03-04T18:03:33.500 INFO:tasks.workunit.client.0.smithi137.stderr: -460> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: crc 2024-03-04T18:03:33.500 INFO:tasks.workunit.client.0.smithi137.stderr: -459> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: secure 2024-03-04T18:03:33.500 INFO:tasks.workunit.client.0.smithi137.stderr: -458> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: crc 2024-03-04T18:03:33.500 INFO:tasks.workunit.client.0.smithi137.stderr: -457> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: secure 2024-03-04T18:03:33.500 INFO:tasks.workunit.client.0.smithi137.stderr: -456> 2024-03-04T18:03:33.174+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.500 INFO:tasks.workunit.client.0.smithi137.stderr: -455> 2024-03-04T18:03:33.174+0000 7f6faaad8980 1 librados: starting msgr at 2024-03-04T18:03:33.500 INFO:tasks.workunit.client.0.smithi137.stderr: -454> 2024-03-04T18:03:33.174+0000 7f6faaad8980 1 librados: starting objecter 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -453> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command objecter_requests hook 0x560b3b8502d0 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -452> 2024-03-04T18:03:33.174+0000 7f6faaad8980 1 librados: setting wanted keys 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -451> 2024-03-04T18:03:33.174+0000 7f6faaad8980 1 librados: calling monclient init 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -450> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: init 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -449> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding auth protocol: cephx 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -448> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding auth protocol: cephx 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -447> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding auth protocol: cephx 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -446> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: secure 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -445> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: crc 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -444> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: secure 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -443> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: crc 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -442> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: secure 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -441> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: crc 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -440> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: crc 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -439> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: secure 2024-03-04T18:03:33.501 INFO:tasks.workunit.client.0.smithi137.stderr: -438> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: crc 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -437> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: secure 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -436> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: crc 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -435> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: secure 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -434> 2024-03-04T18:03:33.174+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -433> 2024-03-04T18:03:33.174+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -432> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command rotate-key hook 0x560b3b904e98 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -431> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: _reopen_session rank -1 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -430> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: _add_conns ranks=[0,2,1] 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -429> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-a con 0x560b3b7b6fd0 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -428> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-c con 0x560b3b84caf0 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -427> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-b con 0x560b3b933ba0 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -426> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -425> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -424> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -423> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): _renew_subs 2024-03-04T18:03:33.502 INFO:tasks.workunit.client.0.smithi137.stderr: -422> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): authenticate will time out at 2446.293275s 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -421> 2024-03-04T18:03:33.174+0000 7f6fa7def640 10 monclient(hunting): get_auth_request con 0x560b3b84caf0 auth_method 0 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -420> 2024-03-04T18:03:33.174+0000 7f6fa7def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -419> 2024-03-04T18:03:33.174+0000 7f6fa7def640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -418> 2024-03-04T18:03:33.174+0000 7f6fa7def640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -417> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request con 0x560b3b7b6fd0 auth_method 0 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -416> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -415> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -414> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -413> 2024-03-04T18:03:33.174+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -412> 2024-03-04T18:03:33.174+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -411> 2024-03-04T18:03:33.174+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -410> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -409> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -408> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.503 INFO:tasks.workunit.client.0.smithi137.stderr: -407> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_done global_id 6288 payload 824 2024-03-04T18:03:33.504 INFO:tasks.workunit.client.0.smithi137.stderr: -406> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient: _finish_hunting 0 2024-03-04T18:03:33.504 INFO:tasks.workunit.client.0.smithi137.stderr: -405> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 1 monclient: found mon.noname-a 2024-03-04T18:03:33.504 INFO:tasks.workunit.client.0.smithi137.stderr: -404> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:03:33.504 INFO:tasks.workunit.client.0.smithi137.stderr: -403> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.504 INFO:tasks.workunit.client.0.smithi137.stderr: -402> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:03:33.504 INFO:tasks.workunit.client.0.smithi137.stderr: -401> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: dump: 2024-03-04T18:03:33.504 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.504 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.504 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.504 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.504 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.504 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.504 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.504 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr: -400> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: _finish_auth 0 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr: -399> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: _check_auth_tickets 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr: -398> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr: -397> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr: -396> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr: -395> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: dump: 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.505 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.506 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.506 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.506 INFO:tasks.workunit.client.0.smithi137.stderr: -394> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 monclient: authenticate success, global_id 6288 2024-03-04T18:03:33.506 INFO:tasks.workunit.client.0.smithi137.stderr: -393> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: _renew_subs 2024-03-04T18:03:33.506 INFO:tasks.workunit.client.0.smithi137.stderr: -392> 2024-03-04T18:03:33.174+0000 7f6e9660c640 4 set_mon_vals no callback set 2024-03-04T18:03:33.506 INFO:tasks.workunit.client.0.smithi137.stderr: -391> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:03:33.506 INFO:tasks.workunit.client.0.smithi137.stderr: -390> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: _renew_subs 2024-03-04T18:03:33.506 INFO:tasks.workunit.client.0.smithi137.stderr: -389> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:03:33.506 INFO:tasks.workunit.client.0.smithi137.stderr: -388> 2024-03-04T18:03:33.174+0000 7f6faaad8980 1 librados: init done 2024-03-04T18:03:33.506 INFO:tasks.workunit.client.0.smithi137.stderr: -387> 2024-03-04T18:03:33.174+0000 7f6e94e09640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:03:33.506 INFO:tasks.workunit.client.0.smithi137.stderr: -386> 2024-03-04T18:03:33.174+0000 7f6e94e09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:03:33.506 INFO:tasks.workunit.client.0.smithi137.stderr: -385> 2024-03-04T18:03:33.174+0000 7f6e94e09640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:03:33.506 INFO:tasks.workunit.client.0.smithi137.stderr: -384> 2024-03-04T18:03:33.178+0000 7f6fa7def640 10 monclient: get_auth_request con 0x7f6e90065970 auth_method 0 2024-03-04T18:03:33.506 INFO:tasks.workunit.client.0.smithi137.stderr: -383> 2024-03-04T18:03:33.178+0000 7f6fa8df1640 10 monclient: get_auth_request con 0x560b3b9323b0 auth_method 0 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr: -382> 2024-03-04T18:03:33.178+0000 7f6fa85f0640 10 monclient: get_auth_request con 0x560b3b94fd40 auth_method 0 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr: -381> 2024-03-04T18:03:33.178+0000 7f6fa7def640 10 monclient: get_auth_request con 0x560b3b954130 auth_method 0 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr: -380> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: build_initial_monmap 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr: -379> 2024-03-04T18:03:33.182+0000 7f6faaad8980 1 build_initial for_mkfs: 0 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr: -378> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: monmap: 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:03:33.507 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -377> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding auth protocol: cephx 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -376> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding auth protocol: cephx 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -375> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding auth protocol: cephx 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -374> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: secure 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -373> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: crc 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -372> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: secure 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -371> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: crc 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -370> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: secure 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -369> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: crc 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -368> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: crc 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -367> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: secure 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -366> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: crc 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -365> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: secure 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -364> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: crc 2024-03-04T18:03:33.508 INFO:tasks.workunit.client.0.smithi137.stderr: -363> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: secure 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -362> 2024-03-04T18:03:33.182+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -361> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command objecter_requests cmddesc objecter_requests hook 0x560b3b95e5c0 EEXIST 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -360> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: init 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -359> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding auth protocol: cephx 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -358> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding auth protocol: cephx 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -357> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding auth protocol: cephx 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -356> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: secure 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -355> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: crc 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -354> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: secure 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -353> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: crc 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -352> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: secure 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -351> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: crc 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -350> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: crc 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -349> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: secure 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -348> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: crc 2024-03-04T18:03:33.509 INFO:tasks.workunit.client.0.smithi137.stderr: -347> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: secure 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -346> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: crc 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -345> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: secure 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -344> 2024-03-04T18:03:33.182+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -343> 2024-03-04T18:03:33.182+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -342> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command rotate-key cmddesc rotate-key hook 0x560b3b957710 EEXIST 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -341> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: _reopen_session rank -1 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -340> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: _add_conns ranks=[0,1,2] 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -339> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-a con 0x560b3b983220 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -338> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-b con 0x560b3b983600 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -337> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-c con 0x560b3b97e7c0 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -336> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -335> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -334> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -333> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): _renew_subs 2024-03-04T18:03:33.510 INFO:tasks.workunit.client.0.smithi137.stderr: -332> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): authenticate will time out at 2446.301679s 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -331> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request con 0x560b3b983220 auth_method 0 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -330> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -329> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -328> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -327> 2024-03-04T18:03:33.182+0000 7f6fa8df1640 10 monclient(hunting): get_auth_request con 0x560b3b97e7c0 auth_method 0 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -326> 2024-03-04T18:03:33.182+0000 7f6fa8df1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -325> 2024-03-04T18:03:33.182+0000 7f6fa8df1640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -324> 2024-03-04T18:03:33.182+0000 7f6fa8df1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -323> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -322> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -321> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -320> 2024-03-04T18:03:33.182+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -319> 2024-03-04T18:03:33.182+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -318> 2024-03-04T18:03:33.182+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.511 INFO:tasks.workunit.client.0.smithi137.stderr: -317> 2024-03-04T18:03:33.182+0000 7f6fa7def640 10 monclient(hunting): get_auth_request con 0x560b3b983600 auth_method 0 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr: -316> 2024-03-04T18:03:33.182+0000 7f6fa7def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr: -315> 2024-03-04T18:03:33.182+0000 7f6fa7def640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr: -314> 2024-03-04T18:03:33.182+0000 7f6fa7def640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr: -313> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_done global_id 6291 payload 824 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr: -312> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient: _finish_hunting 0 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr: -311> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 1 monclient: found mon.noname-a 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr: -310> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr: -309> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr: -308> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr: -307> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: dump: 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.512 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr: -306> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: _finish_auth 0 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr: -305> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: _check_auth_tickets 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr: -304> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr: -303> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr: -302> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr: -301> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: dump: 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.513 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr: -300> 2024-03-04T18:03:33.182+0000 7f6fa95f2640 4 set_mon_vals no callback set 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr: -299> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 monclient: authenticate success, global_id 6291 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr: -298> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: _renew_subs 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr: -297> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr: -296> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: _renew_subs 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr: -295> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr: -294> 2024-03-04T18:03:33.186+0000 7f6e8dffb640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr: -293> 2024-03-04T18:03:33.186+0000 7f6e8dffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr: -292> 2024-03-04T18:03:33.186+0000 7f6e8dffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:03:33.514 INFO:tasks.workunit.client.0.smithi137.stderr: -291> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: get_monmap_and_config 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr: -290> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: build_initial_monmap 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr: -289> 2024-03-04T18:03:33.186+0000 7f6faaad8980 1 build_initial for_mkfs: 0 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr: -288> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: monmap: 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr: -287> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding auth protocol: cephx 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr: -286> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding auth protocol: cephx 2024-03-04T18:03:33.515 INFO:tasks.workunit.client.0.smithi137.stderr: -285> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding auth protocol: cephx 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -284> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -283> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -282> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -281> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -280> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -279> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -278> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -277> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -276> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -275> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -274> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -273> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -272> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient: get_auth_request con 0x7f6e80065880 auth_method 0 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -271> 2024-03-04T18:03:33.186+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.516 INFO:tasks.workunit.client.0.smithi137.stderr: -270> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: init 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -269> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding auth protocol: cephx 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -268> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding auth protocol: cephx 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -267> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding auth protocol: cephx 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -266> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: secure 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -265> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: crc 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -264> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: secure 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -263> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: crc 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -262> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: secure 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -261> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: crc 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -260> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: crc 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -259> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: secure 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -258> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: crc 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -257> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: secure 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -256> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: crc 2024-03-04T18:03:33.517 INFO:tasks.workunit.client.0.smithi137.stderr: -255> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: secure 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -254> 2024-03-04T18:03:33.186+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -253> 2024-03-04T18:03:33.186+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -252> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command rotate-key cmddesc rotate-key hook 0x7ffcb04c5db8 EEXIST 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -251> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: _reopen_session rank -1 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -250> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -249> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-b con 0x560b3b9d5440 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -248> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-a con 0x560b3b9d0950 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -247> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-c con 0x560b3b9da530 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -246> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -245> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -244> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -243> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): _renew_subs 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -242> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): authenticate will time out at 2446.305632s 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -241> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): get_auth_request con 0x560b3b9da530 auth_method 0 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -240> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.518 INFO:tasks.workunit.client.0.smithi137.stderr: -239> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -238> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -237> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient(hunting): get_auth_request con 0x560b3b9d0950 auth_method 0 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -236> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -235> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -234> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -233> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -232> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -231> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -230> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -229> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -228> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -227> 2024-03-04T18:03:33.186+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request con 0x560b3b9d5440 auth_method 0 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -226> 2024-03-04T18:03:33.186+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -225> 2024-03-04T18:03:33.186+0000 7f6fa85f0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.519 INFO:tasks.workunit.client.0.smithi137.stderr: -224> 2024-03-04T18:03:33.186+0000 7f6fa85f0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr: -223> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_done global_id 5978 payload 274 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr: -222> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient: _finish_hunting 0 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr: -221> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 1 monclient: found mon.noname-c 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr: -220> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.137:3301/0 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr: -219> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr: -218> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr: -217> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: dump: 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.520 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr: -216> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: _finish_auth 0 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr: -215> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: _check_auth_tickets 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr: -214> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr: -213> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr: -212> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr: -211> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: dump: 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.521 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr: -210> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 monclient: authenticate success, global_id 5978 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr: -209> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: get_monmap_and_config success 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr: -208> 2024-03-04T18:03:33.186+0000 7f6faaad8980 4 set_mon_vals no callback set 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr: -207> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: shutdown 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr: -206> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: build_initial_monmap 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr: -205> 2024-03-04T18:03:33.186+0000 7f6faaad8980 1 build_initial for_mkfs: 0 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr: -204> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: monmap: 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.522 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr: -203> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding auth protocol: cephx 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr: -202> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding auth protocol: cephx 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr: -201> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding auth protocol: cephx 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr: -200> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr: -199> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr: -198> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr: -197> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr: -196> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr: -195> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr: -194> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr: -193> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr: -192> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.523 INFO:tasks.workunit.client.0.smithi137.stderr: -191> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.524 INFO:tasks.workunit.client.0.smithi137.stderr: -190> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.524 INFO:tasks.workunit.client.0.smithi137.stderr: -189> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.524 INFO:tasks.workunit.client.0.smithi137.stderr: -188> 2024-03-04T18:03:33.186+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.524 INFO:tasks.workunit.client.0.smithi137.stderr: -187> 2024-03-04T18:03:33.186+0000 7f6faaad8980 1 librados: starting msgr at 2024-03-04T18:03:33.524 INFO:tasks.workunit.client.0.smithi137.stderr: -186> 2024-03-04T18:03:33.186+0000 7f6faaad8980 1 librados: starting objecter 2024-03-04T18:03:33.524 INFO:tasks.workunit.client.0.smithi137.stderr: -185> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command objecter_requests cmddesc objecter_requests hook 0x560b3b9a3950 EEXIST 2024-03-04T18:03:33.524 INFO:tasks.workunit.client.0.smithi137.stderr: -184> 2024-03-04T18:03:33.186+0000 7f6faaad8980 1 librados: setting wanted keys 2024-03-04T18:03:33.524 INFO:tasks.workunit.client.0.smithi137.stderr: -183> 2024-03-04T18:03:33.186+0000 7f6faaad8980 1 librados: calling monclient init 2024-03-04T18:03:33.524 INFO:tasks.workunit.client.0.smithi137.stderr: -182> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: init 2024-03-04T18:03:33.524 INFO:tasks.workunit.client.0.smithi137.stderr: -181> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding auth protocol: cephx 2024-03-04T18:03:33.524 INFO:tasks.workunit.client.0.smithi137.stderr: -180> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding auth protocol: cephx 2024-03-04T18:03:33.524 INFO:tasks.workunit.client.0.smithi137.stderr: -179> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding auth protocol: cephx 2024-03-04T18:03:33.524 INFO:tasks.workunit.client.0.smithi137.stderr: -178> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: secure 2024-03-04T18:03:33.524 INFO:tasks.workunit.client.0.smithi137.stderr: -177> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: crc 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -176> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: secure 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -175> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: crc 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -174> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: secure 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -173> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: crc 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -172> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: crc 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -171> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: secure 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -170> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: crc 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -169> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: secure 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -168> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: crc 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -167> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: secure 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -166> 2024-03-04T18:03:33.186+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -165> 2024-03-04T18:03:33.190+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -164> 2024-03-04T18:03:33.190+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command rotate-key cmddesc rotate-key hook 0x560b3b9a0088 EEXIST 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -163> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient: _reopen_session rank -1 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -162> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient: _add_conns ranks=[0,2,1] 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -161> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-a con 0x560b3b9d0950 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:03:33.525 INFO:tasks.workunit.client.0.smithi137.stderr: -160> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-c con 0x560b3b9d5440 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -159> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-b con 0x560b3b9da530 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -158> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -157> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -156> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -155> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): _renew_subs 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -154> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): authenticate will time out at 2446.308304s 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -153> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): get_auth_request con 0x560b3b9d5440 auth_method 0 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -152> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -151> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -150> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -149> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request con 0x560b3b9d0950 auth_method 0 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -148> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -147> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -146> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.526 INFO:tasks.workunit.client.0.smithi137.stderr: -145> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -144> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -143> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -142> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -141> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -140> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -139> 2024-03-04T18:03:33.190+0000 7f6fa8df1640 10 monclient(hunting): get_auth_request con 0x560b3b9da530 auth_method 0 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -138> 2024-03-04T18:03:33.190+0000 7f6fa8df1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -137> 2024-03-04T18:03:33.190+0000 7f6fa8df1640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -136> 2024-03-04T18:03:33.190+0000 7f6fa8df1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -135> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): handle_auth_done global_id 5981 payload 824 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -134> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient: _finish_hunting 0 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -133> 2024-03-04T18:03:33.190+0000 7f6fa7def640 1 monclient: found mon.noname-c 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -132> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.137:3301/0 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -131> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -130> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr: -129> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: dump: 2024-03-04T18:03:33.527 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.528 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.528 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.528 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.528 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.528 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.528 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.528 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.528 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.528 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.528 INFO:tasks.workunit.client.0.smithi137.stderr: -128> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: _finish_auth 0 2024-03-04T18:03:33.528 INFO:tasks.workunit.client.0.smithi137.stderr: -127> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: _check_auth_tickets 2024-03-04T18:03:33.528 INFO:tasks.workunit.client.0.smithi137.stderr: -126> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:03:33.528 INFO:tasks.workunit.client.0.smithi137.stderr: -125> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.528 INFO:tasks.workunit.client.0.smithi137.stderr: -124> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr: -123> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: dump: 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr: -122> 2024-03-04T18:03:33.190+0000 7f6faaad8980 5 monclient: authenticate success, global_id 5981 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr: -121> 2024-03-04T18:03:33.190+0000 7f6e7f7fe640 4 set_mon_vals no callback set 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr: -120> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient: _renew_subs 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr: -119> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.137:3301/0 2024-03-04T18:03:33.529 INFO:tasks.workunit.client.0.smithi137.stderr: -118> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient: _renew_subs 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -117> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.137:3301/0 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -116> 2024-03-04T18:03:33.190+0000 7f6faaad8980 1 librados: init done 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -115> 2024-03-04T18:03:33.190+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command cr dump hook 0x560b3b9c4d58 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -114> 2024-03-04T18:03:33.190+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -113> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -112> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -111> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -110> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient: get_auth_request con 0x7f6e70065940 auth_method 0 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -109> 2024-03-04T18:03:33.190+0000 7f6fa8df1640 10 monclient: get_auth_request con 0x560b3b9ef650 auth_method 0 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -108> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -107> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 realm 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -106> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -105> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -104> 2024-03-04T18:03:33.194+0000 7f6faaad8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:33.530 INFO:tasks.workunit.client.0.smithi137.stderr: -103> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -102> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -101> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -100> 2024-03-04T18:03:33.194+0000 7f6fa85f0640 10 monclient: get_auth_request con 0x560b3b9f3810 auth_method 0 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -99> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -98> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -97> 2024-03-04T18:03:33.194+0000 7f6fa7def640 10 monclient: get_auth_request con 0x560b3b9f7070 auth_method 0 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -96> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -95> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 searching for the correct realm 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -94> 2024-03-04T18:03:33.194+0000 7f6fa8df1640 10 monclient: get_auth_request con 0x560b3b9fbdb0 auth_method 0 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -93> 2024-03-04T18:03:33.198+0000 7f6fa85f0640 10 monclient: get_auth_request con 0x7f6fa00a9070 auth_method 0 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -92> 2024-03-04T18:03:33.198+0000 7f6fa7def640 10 monclient: get_auth_request con 0x7f6f98064b30 auth_method 0 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -91> 2024-03-04T18:03:33.202+0000 7f6fa8df1640 10 monclient: get_auth_request con 0x7f6fa00c9400 auth_method 0 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -90> 2024-03-04T18:03:33.202+0000 7f6fa8df1640 0 -- 172.21.15.137:0/2758601501 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f6fa00c9400 msgr2=0x7f6fa00c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -89> 2024-03-04T18:03:33.394+0000 7f6fa85f0640 10 monclient: get_auth_request con 0x7f6e70065940 auth_method 0 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -88> 2024-03-04T18:03:33.406+0000 7f6fa8df1640 10 monclient: get_auth_request con 0x7f6fa00c9400 auth_method 0 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -87> 2024-03-04T18:03:33.410+0000 7f6fa85f0640 10 monclient: get_auth_request con 0x7f6f9806c360 auth_method 0 2024-03-04T18:03:33.531 INFO:tasks.workunit.client.0.smithi137.stderr: -86> 2024-03-04T18:03:33.414+0000 7f6fa7def640 0 -- 172.21.15.137:0/2758601501 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x560b3b9f7070 msgr2=0x560b3b9f9480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -85> 2024-03-04T18:03:33.414+0000 7f6e7dffb640 1 client.5981.objecter ms_handle_reset 0x560b3b9f7070 session 0x560b3b9f2ba0 osd.3 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -84> 2024-03-04T18:03:33.414+0000 7f6e7dffb640 10 monclient: _renew_subs 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -83> 2024-03-04T18:03:33.414+0000 7f6e7dffb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.137:3301/0 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -82> 2024-03-04T18:03:33.418+0000 7f6fa7def640 10 monclient: get_auth_request con 0x7f6e70075460 auth_method 0 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -81> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -80> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -79> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -78> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -77> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -76> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -75> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -74> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -73> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -72> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:33.532 INFO:tasks.workunit.client.0.smithi137.stderr: -71> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -70> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -69> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 zone default found 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -68> 2024-03-04T18:03:33.430+0000 7f6faaad8980 4 Realm: () 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -67> 2024-03-04T18:03:33.430+0000 7f6faaad8980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -66> 2024-03-04T18:03:33.430+0000 7f6faaad8980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -65> 2024-03-04T18:03:33.430+0000 7f6faaad8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -64> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 zonegroup default 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -63> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -62> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -61> 2024-03-04T18:03:33.434+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -60> 2024-03-04T18:03:33.434+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -59> 2024-03-04T18:03:33.434+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -58> 2024-03-04T18:03:33.434+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -57> 2024-03-04T18:03:33.434+0000 7f6faaad8980 20 started sync module instance, tier type = 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -56> 2024-03-04T18:03:33.434+0000 7f6faaad8980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:33.533 INFO:tasks.workunit.client.0.smithi137.stderr: -55> 2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=0 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -54> 2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=1 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -53> 2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=2 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -52> 2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=3 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -51> 2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=4 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -50> 2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=5 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -49> 2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=6 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -48> 2024-03-04T18:03:33.458+0000 7f6faaad8980 20 add_watcher() i=7 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -47> 2024-03-04T18:03:33.458+0000 7f6faaad8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -46> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding auth protocol: cephx 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -45> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding auth protocol: cephx 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -44> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding auth protocol: cephx 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -43> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: secure 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -42> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: crc 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -41> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: secure 2024-03-04T18:03:33.534 INFO:tasks.workunit.client.0.smithi137.stderr: -40> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: crc 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -39> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: secure 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -38> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: crc 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -37> 2024-03-04T18:03:33.462+0000 7f6e5efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -36> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: crc 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -35> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: secure 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -34> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: crc 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -33> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: secure 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -32> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: crc 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -31> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: secure 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -30> 2024-03-04T18:03:33.462+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -29> 2024-03-04T18:03:33.462+0000 7f6faaad8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -28> 2024-03-04T18:03:33.462+0000 7f6faaad8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -27> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command cache list hook 0x560b3b9c9fe0 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -26> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command cache inspect hook 0x560b3b9c9fe0 2024-03-04T18:03:33.535 INFO:tasks.workunit.client.0.smithi137.stderr: -25> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command cache erase hook 0x560b3b9c9fe0 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -24> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command cache zap hook 0x560b3b9c9fe0 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -23> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 note: GC not initialized 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -22> 2024-03-04T18:03:33.462+0000 7f6e5dfdb640 20 reqs_thread_entry: start 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -21> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command sync trace show hook 0x7f6fa00047f0 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -20> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command sync trace history hook 0x7f6fa00047f0 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -19> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command sync trace active hook 0x7f6fa00047f0 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -18> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command sync trace active_short hook 0x7f6fa00047f0 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -17> 2024-03-04T18:03:33.462+0000 7f6faaad8980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -16> 2024-03-04T18:03:33.466+0000 7f6faaad8980 20 Filter name: none 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -15> 2024-03-04T18:03:33.466+0000 7f6dcf7fe640 20 reqs_thread_entry: start 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -14> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -13> 2024-03-04T18:03:33.466+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -12> 2024-03-04T18:03:33.466+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -11> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-03-04T18:03:33.536 INFO:tasks.workunit.client.0.smithi137.stderr: -10> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: -9> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 : miss 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: -8> 2024-03-04T18:03:33.466+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: -7> 2024-03-04T18:03:33.466+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: -6> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 info.flags=0x17 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: -5> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 to cache LRU end 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: -4> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: -3> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: -2> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: -1> 2024-03-04T18:03:33.470+0000 7f6faaad8980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f6faaad8980 time 2024-03-04T18:03:33.473595+0000 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f6fae3add7b] 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f6fae3adf2f] 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560b39462281] 2024-03-04T18:03:33.537 INFO:tasks.workunit.client.0.smithi137.stderr: 4: radosgw-admin(+0x8c3b2a) [0x560b39469b2a] 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x560b3946ec93] 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: 6: main() 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6fad218d90] 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: 8: __libc_start_main() 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: 9: _start() 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: 0> 2024-03-04T18:03:33.474+0000 7f6faaad8980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7f6faaad8980 thread_name:radosgw-admin 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f6fad231520] 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:03:33.538 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f6fae3addd5] 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f6fae3adf2f] 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560b39462281] 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: 8: radosgw-admin(+0x8c3b2a) [0x560b39469b2a] 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x560b3946ec93] 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: 10: main() 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6fad218d90] 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: 12: __libc_start_main() 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: 13: _start() 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr:--- logging levels --- 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 none 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 lockdep 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 context 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 crush 2024-03-04T18:03:33.539 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds 2024-03-04T18:03:33.540 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_balancer 2024-03-04T18:03:33.540 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_locker 2024-03-04T18:03:33.540 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log 2024-03-04T18:03:33.540 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log_expire 2024-03-04T18:03:33.540 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_migrator 2024-03-04T18:03:33.540 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 buffer 2024-03-04T18:03:33.540 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 timer 2024-03-04T18:03:33.540 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 filer 2024-03-04T18:03:33.540 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 striper 2024-03-04T18:03:33.540 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 objecter 2024-03-04T18:03:33.540 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rados 2024-03-04T18:03:33.540 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd 2024-03-04T18:03:33.540 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_mirror 2024-03-04T18:03:33.540 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_replay 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_pwl 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 journaler 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objectcacher 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 client 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 osd 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 optracker 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objclass 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 filestore 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 journal 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 ms 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mon 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 0/10 monc 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 paxos 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 tp 2024-03-04T18:03:33.541 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 auth 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 crypto 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 finisher 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 reserver 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 heartbeatmap 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 perfcounter 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 20/20 rgw 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_sync 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_datacache 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_access 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_flight 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 javaclient 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 asok 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 throttle 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 refs 2024-03-04T18:03:33.542 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 compressor 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluestore 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluefs 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 bdev 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 kstore 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 4/ 5 rocksdb 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 fuse 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 2/ 5 mgr 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mgrc 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 dpdk 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 eventtrace 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 prioritycache 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 test 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephsqlite 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore 2024-03-04T18:03:33.543 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_onode 2024-03-04T18:03:33.544 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_odata 2024-03-04T18:03:33.544 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_omap 2024-03-04T18:03:33.544 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_tm 2024-03-04T18:03:33.544 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_t 2024-03-04T18:03:33.544 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:03:33.544 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_epm 2024-03-04T18:03:33.544 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_lba 2024-03-04T18:03:33.544 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:03:33.544 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cache 2024-03-04T18:03:33.544 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_journal 2024-03-04T18:03:33.544 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_device 2024-03-04T18:03:33.544 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_backref 2024-03-04T18:03:33.544 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 alienstore 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mclock 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cyanstore 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 ceph_exporter 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 memstore 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 trace 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: -2/-2 (syslog threshold) 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: 99/99 (stderr threshold) 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6dcf7fe640 / http_manager 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6e5dfdb640 / http_manager 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6e5efdd640 / rgw_dt_lg_renew 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6e7dffb640 / ms_dispatch 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6e7f7fe640 / io_context_pool 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6e8dffb640 / ms_dispatch 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6e94e09640 / ms_dispatch 2024-03-04T18:03:33.545 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6e9660c640 / io_context_pool 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6fa7def640 / msgr-worker-2 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6fa85f0640 / msgr-worker-1 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6fa8df1640 / msgr-worker-0 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6fa95f2640 / io_context_pool 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6faaad8980 / radosgw-admin 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: max_recent 500 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: max_new 1000 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: log_file /var/log/ceph/ceph-client.admin.81188.log 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr:--- end dump of recent events --- 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr:--- begin dump of recent events --- 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: -499> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: -498> 2024-03-04T18:03:33.170+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: -497> 2024-03-04T18:03:33.170+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: -496> 2024-03-04T18:03:33.170+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: -495> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: -494> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.546 INFO:tasks.workunit.client.0.smithi137.stderr: -493> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr: -492> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_done global_id 6285 payload 274 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr: -491> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 10 monclient: _finish_hunting 0 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr: -490> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 1 monclient: found mon.noname-a 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr: -489> 2024-03-04T18:03:33.170+0000 7f6fa8df1640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr: -488> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr: -487> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr: -486> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: dump: 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr: -485> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: _finish_auth 0 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr: -484> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: _check_auth_tickets 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr: -483> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr: -482> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr: -481> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr: -480> 2024-03-04T18:03:33.170+0000 7f6e9660c640 10 monclient: dump: 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.548 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr: -479> 2024-03-04T18:03:33.170+0000 7f6faaad8980 5 monclient: authenticate success, global_id 6285 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr: -478> 2024-03-04T18:03:33.170+0000 7f6faaad8980 10 monclient: get_monmap_and_config success 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr: -477> 2024-03-04T18:03:33.170+0000 7f6faaad8980 4 set_mon_vals no callback set 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr: -476> 2024-03-04T18:03:33.170+0000 7f6faaad8980 10 monclient: shutdown 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr: -475> 2024-03-04T18:03:33.170+0000 7f6faaad8980 5 asok(0x560b3b71f130) unregister_commands rotate-key 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr: -474> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: build_initial_monmap 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr: -473> 2024-03-04T18:03:33.174+0000 7f6faaad8980 1 build_initial for_mkfs: 0 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr: -472> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: monmap: 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:03:33.549 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr: -471> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding auth protocol: cephx 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr: -470> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding auth protocol: cephx 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr: -469> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding auth protocol: cephx 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr: -468> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: secure 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr: -467> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: crc 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr: -466> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: secure 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr: -465> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: crc 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr: -464> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: secure 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr: -463> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: crc 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr: -462> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: crc 2024-03-04T18:03:33.550 INFO:tasks.workunit.client.0.smithi137.stderr: -461> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: secure 2024-03-04T18:03:33.551 INFO:tasks.workunit.client.0.smithi137.stderr: -460> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: crc 2024-03-04T18:03:33.551 INFO:tasks.workunit.client.0.smithi137.stderr: -459> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: secure 2024-03-04T18:03:33.551 INFO:tasks.workunit.client.0.smithi137.stderr: -458> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: crc 2024-03-04T18:03:33.551 INFO:tasks.workunit.client.0.smithi137.stderr: -457> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b906bb0) adding con mode: secure 2024-03-04T18:03:33.551 INFO:tasks.workunit.client.0.smithi137.stderr: -456> 2024-03-04T18:03:33.174+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.551 INFO:tasks.workunit.client.0.smithi137.stderr: -455> 2024-03-04T18:03:33.174+0000 7f6faaad8980 1 librados: starting msgr at 2024-03-04T18:03:33.551 INFO:tasks.workunit.client.0.smithi137.stderr: -454> 2024-03-04T18:03:33.174+0000 7f6faaad8980 1 librados: starting objecter 2024-03-04T18:03:33.551 INFO:tasks.workunit.client.0.smithi137.stderr: -453> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command objecter_requests hook 0x560b3b8502d0 2024-03-04T18:03:33.551 INFO:tasks.workunit.client.0.smithi137.stderr: -452> 2024-03-04T18:03:33.174+0000 7f6faaad8980 1 librados: setting wanted keys 2024-03-04T18:03:33.551 INFO:tasks.workunit.client.0.smithi137.stderr: -451> 2024-03-04T18:03:33.174+0000 7f6faaad8980 1 librados: calling monclient init 2024-03-04T18:03:33.551 INFO:tasks.workunit.client.0.smithi137.stderr: -450> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: init 2024-03-04T18:03:33.551 INFO:tasks.workunit.client.0.smithi137.stderr: -449> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding auth protocol: cephx 2024-03-04T18:03:33.551 INFO:tasks.workunit.client.0.smithi137.stderr: -448> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding auth protocol: cephx 2024-03-04T18:03:33.551 INFO:tasks.workunit.client.0.smithi137.stderr: -447> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding auth protocol: cephx 2024-03-04T18:03:33.552 INFO:tasks.workunit.client.0.smithi137.stderr: -446> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: secure 2024-03-04T18:03:33.552 INFO:tasks.workunit.client.0.smithi137.stderr: -445> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: crc 2024-03-04T18:03:33.552 INFO:tasks.workunit.client.0.smithi137.stderr: -444> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: secure 2024-03-04T18:03:33.552 INFO:tasks.workunit.client.0.smithi137.stderr: -443> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: crc 2024-03-04T18:03:33.552 INFO:tasks.workunit.client.0.smithi137.stderr: -442> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: secure 2024-03-04T18:03:33.552 INFO:tasks.workunit.client.0.smithi137.stderr: -441> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: crc 2024-03-04T18:03:33.552 INFO:tasks.workunit.client.0.smithi137.stderr: -440> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: crc 2024-03-04T18:03:33.552 INFO:tasks.workunit.client.0.smithi137.stderr: -439> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: secure 2024-03-04T18:03:33.552 INFO:tasks.workunit.client.0.smithi137.stderr: -438> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: crc 2024-03-04T18:03:33.552 INFO:tasks.workunit.client.0.smithi137.stderr: -437> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: secure 2024-03-04T18:03:33.552 INFO:tasks.workunit.client.0.smithi137.stderr: -436> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: crc 2024-03-04T18:03:33.552 INFO:tasks.workunit.client.0.smithi137.stderr: -435> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 AuthRegistry(0x560b3b904d50) adding con mode: secure 2024-03-04T18:03:33.552 INFO:tasks.workunit.client.0.smithi137.stderr: -434> 2024-03-04T18:03:33.174+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -433> 2024-03-04T18:03:33.174+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -432> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command rotate-key hook 0x560b3b904e98 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -431> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: _reopen_session rank -1 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -430> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: _add_conns ranks=[0,2,1] 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -429> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-a con 0x560b3b7b6fd0 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -428> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-c con 0x560b3b84caf0 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -427> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-b con 0x560b3b933ba0 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -426> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -425> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -424> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -423> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): _renew_subs 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -422> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient(hunting): authenticate will time out at 2446.293275s 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -421> 2024-03-04T18:03:33.174+0000 7f6fa7def640 10 monclient(hunting): get_auth_request con 0x560b3b84caf0 auth_method 0 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -420> 2024-03-04T18:03:33.174+0000 7f6fa7def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -419> 2024-03-04T18:03:33.174+0000 7f6fa7def640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.553 INFO:tasks.workunit.client.0.smithi137.stderr: -418> 2024-03-04T18:03:33.174+0000 7f6fa7def640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.554 INFO:tasks.workunit.client.0.smithi137.stderr: -417> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request con 0x560b3b7b6fd0 auth_method 0 2024-03-04T18:03:33.554 INFO:tasks.workunit.client.0.smithi137.stderr: -416> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.554 INFO:tasks.workunit.client.0.smithi137.stderr: -415> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.554 INFO:tasks.workunit.client.0.smithi137.stderr: -414> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.554 INFO:tasks.workunit.client.0.smithi137.stderr: -413> 2024-03-04T18:03:33.174+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.554 INFO:tasks.workunit.client.0.smithi137.stderr: -412> 2024-03-04T18:03:33.174+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.554 INFO:tasks.workunit.client.0.smithi137.stderr: -411> 2024-03-04T18:03:33.174+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.554 INFO:tasks.workunit.client.0.smithi137.stderr: -410> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.554 INFO:tasks.workunit.client.0.smithi137.stderr: -409> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.554 INFO:tasks.workunit.client.0.smithi137.stderr: -408> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.554 INFO:tasks.workunit.client.0.smithi137.stderr: -407> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_done global_id 6288 payload 824 2024-03-04T18:03:33.554 INFO:tasks.workunit.client.0.smithi137.stderr: -406> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient: _finish_hunting 0 2024-03-04T18:03:33.554 INFO:tasks.workunit.client.0.smithi137.stderr: -405> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 1 monclient: found mon.noname-a 2024-03-04T18:03:33.554 INFO:tasks.workunit.client.0.smithi137.stderr: -404> 2024-03-04T18:03:33.174+0000 7f6fa85f0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr: -403> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr: -402> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr: -401> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: dump: 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr: -400> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: _finish_auth 0 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr: -399> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: _check_auth_tickets 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr: -398> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:03:33.555 INFO:tasks.workunit.client.0.smithi137.stderr: -397> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr: -396> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr: -395> 2024-03-04T18:03:33.174+0000 7f6e94e09640 10 monclient: dump: 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr: -394> 2024-03-04T18:03:33.174+0000 7f6faaad8980 5 monclient: authenticate success, global_id 6288 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr: -393> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: _renew_subs 2024-03-04T18:03:33.556 INFO:tasks.workunit.client.0.smithi137.stderr: -392> 2024-03-04T18:03:33.174+0000 7f6e9660c640 4 set_mon_vals no callback set 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr: -391> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr: -390> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: _renew_subs 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr: -389> 2024-03-04T18:03:33.174+0000 7f6faaad8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr: -388> 2024-03-04T18:03:33.174+0000 7f6faaad8980 1 librados: init done 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr: -387> 2024-03-04T18:03:33.174+0000 7f6e94e09640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr: -386> 2024-03-04T18:03:33.174+0000 7f6e94e09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr: -385> 2024-03-04T18:03:33.174+0000 7f6e94e09640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr: -384> 2024-03-04T18:03:33.178+0000 7f6fa7def640 10 monclient: get_auth_request con 0x7f6e90065970 auth_method 0 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr: -383> 2024-03-04T18:03:33.178+0000 7f6fa8df1640 10 monclient: get_auth_request con 0x560b3b9323b0 auth_method 0 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr: -382> 2024-03-04T18:03:33.178+0000 7f6fa85f0640 10 monclient: get_auth_request con 0x560b3b94fd40 auth_method 0 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr: -381> 2024-03-04T18:03:33.178+0000 7f6fa7def640 10 monclient: get_auth_request con 0x560b3b954130 auth_method 0 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr: -380> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: build_initial_monmap 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr: -379> 2024-03-04T18:03:33.182+0000 7f6faaad8980 1 build_initial for_mkfs: 0 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr: -378> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: monmap: 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:03:33.557 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr: -377> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding auth protocol: cephx 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr: -376> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding auth protocol: cephx 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr: -375> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding auth protocol: cephx 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr: -374> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: secure 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr: -373> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: crc 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr: -372> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: secure 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr: -371> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: crc 2024-03-04T18:03:33.558 INFO:tasks.workunit.client.0.smithi137.stderr: -370> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: secure 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -369> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: crc 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -368> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: crc 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -367> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: secure 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -366> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: crc 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -365> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: secure 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -364> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: crc 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -363> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b958e70) adding con mode: secure 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -362> 2024-03-04T18:03:33.182+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -361> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command objecter_requests cmddesc objecter_requests hook 0x560b3b95e5c0 EEXIST 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -360> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: init 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -359> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding auth protocol: cephx 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -358> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding auth protocol: cephx 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -357> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding auth protocol: cephx 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -356> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: secure 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -355> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: crc 2024-03-04T18:03:33.559 INFO:tasks.workunit.client.0.smithi137.stderr: -354> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: secure 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -353> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: crc 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -352> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: secure 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -351> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: crc 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -350> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: crc 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -349> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: secure 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -348> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: crc 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -347> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: secure 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -346> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: crc 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -345> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9575c8) adding con mode: secure 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -344> 2024-03-04T18:03:33.182+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -343> 2024-03-04T18:03:33.182+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -342> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command rotate-key cmddesc rotate-key hook 0x560b3b957710 EEXIST 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -341> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: _reopen_session rank -1 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -340> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: _add_conns ranks=[0,1,2] 2024-03-04T18:03:33.560 INFO:tasks.workunit.client.0.smithi137.stderr: -339> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-a con 0x560b3b983220 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -338> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-b con 0x560b3b983600 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -337> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-c con 0x560b3b97e7c0 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -336> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -335> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -334> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -333> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): _renew_subs 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -332> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient(hunting): authenticate will time out at 2446.301679s 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -331> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request con 0x560b3b983220 auth_method 0 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -330> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -329> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -328> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -327> 2024-03-04T18:03:33.182+0000 7f6fa8df1640 10 monclient(hunting): get_auth_request con 0x560b3b97e7c0 auth_method 0 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -326> 2024-03-04T18:03:33.182+0000 7f6fa8df1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -325> 2024-03-04T18:03:33.182+0000 7f6fa8df1640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -324> 2024-03-04T18:03:33.182+0000 7f6fa8df1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.561 INFO:tasks.workunit.client.0.smithi137.stderr: -323> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.562 INFO:tasks.workunit.client.0.smithi137.stderr: -322> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.562 INFO:tasks.workunit.client.0.smithi137.stderr: -321> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.562 INFO:tasks.workunit.client.0.smithi137.stderr: -320> 2024-03-04T18:03:33.182+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.562 INFO:tasks.workunit.client.0.smithi137.stderr: -319> 2024-03-04T18:03:33.182+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.562 INFO:tasks.workunit.client.0.smithi137.stderr: -318> 2024-03-04T18:03:33.182+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.562 INFO:tasks.workunit.client.0.smithi137.stderr: -317> 2024-03-04T18:03:33.182+0000 7f6fa7def640 10 monclient(hunting): get_auth_request con 0x560b3b983600 auth_method 0 2024-03-04T18:03:33.562 INFO:tasks.workunit.client.0.smithi137.stderr: -316> 2024-03-04T18:03:33.182+0000 7f6fa7def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.562 INFO:tasks.workunit.client.0.smithi137.stderr: -315> 2024-03-04T18:03:33.182+0000 7f6fa7def640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.562 INFO:tasks.workunit.client.0.smithi137.stderr: -314> 2024-03-04T18:03:33.182+0000 7f6fa7def640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.562 INFO:tasks.workunit.client.0.smithi137.stderr: -313> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_done global_id 6291 payload 824 2024-03-04T18:03:33.562 INFO:tasks.workunit.client.0.smithi137.stderr: -312> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient: _finish_hunting 0 2024-03-04T18:03:33.562 INFO:tasks.workunit.client.0.smithi137.stderr: -311> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 1 monclient: found mon.noname-a 2024-03-04T18:03:33.562 INFO:tasks.workunit.client.0.smithi137.stderr: -310> 2024-03-04T18:03:33.182+0000 7f6fa85f0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:03:33.562 INFO:tasks.workunit.client.0.smithi137.stderr: -309> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr: -308> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr: -307> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: dump: 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr: -306> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: _finish_auth 0 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr: -305> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: _check_auth_tickets 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr: -304> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr: -303> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.563 INFO:tasks.workunit.client.0.smithi137.stderr: -302> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr: -301> 2024-03-04T18:03:33.182+0000 7f6e8dffb640 10 monclient: dump: 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr: -300> 2024-03-04T18:03:33.182+0000 7f6fa95f2640 4 set_mon_vals no callback set 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr: -299> 2024-03-04T18:03:33.182+0000 7f6faaad8980 5 monclient: authenticate success, global_id 6291 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr: -298> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: _renew_subs 2024-03-04T18:03:33.564 INFO:tasks.workunit.client.0.smithi137.stderr: -297> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr: -296> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: _renew_subs 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr: -295> 2024-03-04T18:03:33.182+0000 7f6faaad8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr: -294> 2024-03-04T18:03:33.186+0000 7f6e8dffb640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr: -293> 2024-03-04T18:03:33.186+0000 7f6e8dffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr: -292> 2024-03-04T18:03:33.186+0000 7f6e8dffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr: -291> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: get_monmap_and_config 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr: -290> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: build_initial_monmap 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr: -289> 2024-03-04T18:03:33.186+0000 7f6faaad8980 1 build_initial for_mkfs: 0 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr: -288> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: monmap: 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.565 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: -287> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding auth protocol: cephx 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: -286> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding auth protocol: cephx 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: -285> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding auth protocol: cephx 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: -284> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: -283> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: -282> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: -281> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: -280> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: -279> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: -278> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: -277> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: -276> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: -275> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.566 INFO:tasks.workunit.client.0.smithi137.stderr: -274> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -273> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -272> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient: get_auth_request con 0x7f6e80065880 auth_method 0 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -271> 2024-03-04T18:03:33.186+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -270> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: init 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -269> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding auth protocol: cephx 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -268> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding auth protocol: cephx 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -267> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding auth protocol: cephx 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -266> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: secure 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -265> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: crc 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -264> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: secure 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -263> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: crc 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -262> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: secure 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -261> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: crc 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -260> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: crc 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -259> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: secure 2024-03-04T18:03:33.567 INFO:tasks.workunit.client.0.smithi137.stderr: -258> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: crc 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -257> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: secure 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -256> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: crc 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -255> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c5c70) adding con mode: secure 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -254> 2024-03-04T18:03:33.186+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -253> 2024-03-04T18:03:33.186+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -252> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command rotate-key cmddesc rotate-key hook 0x7ffcb04c5db8 EEXIST 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -251> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: _reopen_session rank -1 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -250> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -249> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-b con 0x560b3b9d5440 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -248> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-a con 0x560b3b9d0950 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -247> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-c con 0x560b3b9da530 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -246> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -245> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -244> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.568 INFO:tasks.workunit.client.0.smithi137.stderr: -243> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): _renew_subs 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -242> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient(hunting): authenticate will time out at 2446.305632s 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -241> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): get_auth_request con 0x560b3b9da530 auth_method 0 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -240> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -239> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -238> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -237> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient(hunting): get_auth_request con 0x560b3b9d0950 auth_method 0 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -236> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -235> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -234> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -233> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -232> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -231> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -230> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -229> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.569 INFO:tasks.workunit.client.0.smithi137.stderr: -228> 2024-03-04T18:03:33.186+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr: -227> 2024-03-04T18:03:33.186+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request con 0x560b3b9d5440 auth_method 0 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr: -226> 2024-03-04T18:03:33.186+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr: -225> 2024-03-04T18:03:33.186+0000 7f6fa85f0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr: -224> 2024-03-04T18:03:33.186+0000 7f6fa85f0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr: -223> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient(hunting): handle_auth_done global_id 5978 payload 274 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr: -222> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient: _finish_hunting 0 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr: -221> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 1 monclient: found mon.noname-c 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr: -220> 2024-03-04T18:03:33.186+0000 7f6fa8df1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.137:3301/0 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr: -219> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr: -218> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr: -217> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: dump: 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.570 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr: -216> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: _finish_auth 0 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr: -215> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: _check_auth_tickets 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr: -214> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr: -213> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr: -212> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr: -211> 2024-03-04T18:03:33.186+0000 7f6e7f7fe640 10 monclient: dump: 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.571 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.572 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.572 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.572 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.572 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.572 INFO:tasks.workunit.client.0.smithi137.stderr: -210> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 monclient: authenticate success, global_id 5978 2024-03-04T18:03:33.572 INFO:tasks.workunit.client.0.smithi137.stderr: -209> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: get_monmap_and_config success 2024-03-04T18:03:33.572 INFO:tasks.workunit.client.0.smithi137.stderr: -208> 2024-03-04T18:03:33.186+0000 7f6faaad8980 4 set_mon_vals no callback set 2024-03-04T18:03:33.572 INFO:tasks.workunit.client.0.smithi137.stderr: -207> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: shutdown 2024-03-04T18:03:33.572 INFO:tasks.workunit.client.0.smithi137.stderr: -206> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: build_initial_monmap 2024-03-04T18:03:33.572 INFO:tasks.workunit.client.0.smithi137.stderr: -205> 2024-03-04T18:03:33.186+0000 7f6faaad8980 1 build_initial for_mkfs: 0 2024-03-04T18:03:33.572 INFO:tasks.workunit.client.0.smithi137.stderr: -204> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: monmap: 2024-03-04T18:03:33.572 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:03:33.572 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:03:33.572 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr: -203> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding auth protocol: cephx 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr: -202> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding auth protocol: cephx 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr: -201> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding auth protocol: cephx 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr: -200> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr: -199> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr: -198> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr: -197> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr: -196> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.573 INFO:tasks.workunit.client.0.smithi137.stderr: -195> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -194> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -193> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -192> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -191> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -190> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: crc 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -189> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b9a0d70) adding con mode: secure 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -188> 2024-03-04T18:03:33.186+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -187> 2024-03-04T18:03:33.186+0000 7f6faaad8980 1 librados: starting msgr at 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -186> 2024-03-04T18:03:33.186+0000 7f6faaad8980 1 librados: starting objecter 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -185> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command objecter_requests cmddesc objecter_requests hook 0x560b3b9a3950 EEXIST 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -184> 2024-03-04T18:03:33.186+0000 7f6faaad8980 1 librados: setting wanted keys 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -183> 2024-03-04T18:03:33.186+0000 7f6faaad8980 1 librados: calling monclient init 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -182> 2024-03-04T18:03:33.186+0000 7f6faaad8980 10 monclient: init 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -181> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding auth protocol: cephx 2024-03-04T18:03:33.574 INFO:tasks.workunit.client.0.smithi137.stderr: -180> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding auth protocol: cephx 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -179> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding auth protocol: cephx 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -178> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: secure 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -177> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: crc 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -176> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: secure 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -175> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: crc 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -174> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: secure 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -173> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: crc 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -172> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: crc 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -171> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: secure 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -170> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: crc 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -169> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: secure 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -168> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: crc 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -167> 2024-03-04T18:03:33.186+0000 7f6faaad8980 5 AuthRegistry(0x560b3b99ff40) adding con mode: secure 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -166> 2024-03-04T18:03:33.186+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -165> 2024-03-04T18:03:33.190+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.575 INFO:tasks.workunit.client.0.smithi137.stderr: -164> 2024-03-04T18:03:33.190+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command rotate-key cmddesc rotate-key hook 0x560b3b9a0088 EEXIST 2024-03-04T18:03:33.576 INFO:tasks.workunit.client.0.smithi137.stderr: -163> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient: _reopen_session rank -1 2024-03-04T18:03:33.576 INFO:tasks.workunit.client.0.smithi137.stderr: -162> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient: _add_conns ranks=[0,2,1] 2024-03-04T18:03:33.576 INFO:tasks.workunit.client.0.smithi137.stderr: -161> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-a con 0x560b3b9d0950 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:03:33.576 INFO:tasks.workunit.client.0.smithi137.stderr: -160> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-c con 0x560b3b9d5440 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:03:33.576 INFO:tasks.workunit.client.0.smithi137.stderr: -159> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): picked mon.noname-b con 0x560b3b9da530 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:03:33.576 INFO:tasks.workunit.client.0.smithi137.stderr: -158> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.576 INFO:tasks.workunit.client.0.smithi137.stderr: -157> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.576 INFO:tasks.workunit.client.0.smithi137.stderr: -156> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:03:33.576 INFO:tasks.workunit.client.0.smithi137.stderr: -155> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): _renew_subs 2024-03-04T18:03:33.576 INFO:tasks.workunit.client.0.smithi137.stderr: -154> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient(hunting): authenticate will time out at 2446.308304s 2024-03-04T18:03:33.576 INFO:tasks.workunit.client.0.smithi137.stderr: -153> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): get_auth_request con 0x560b3b9d5440 auth_method 0 2024-03-04T18:03:33.576 INFO:tasks.workunit.client.0.smithi137.stderr: -152> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.576 INFO:tasks.workunit.client.0.smithi137.stderr: -151> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.576 INFO:tasks.workunit.client.0.smithi137.stderr: -150> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -149> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request con 0x560b3b9d0950 auth_method 0 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -148> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -147> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -146> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -145> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -144> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -143> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -142> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -141> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -140> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -139> 2024-03-04T18:03:33.190+0000 7f6fa8df1640 10 monclient(hunting): get_auth_request con 0x560b3b9da530 auth_method 0 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -138> 2024-03-04T18:03:33.190+0000 7f6fa8df1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -137> 2024-03-04T18:03:33.190+0000 7f6fa8df1640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -136> 2024-03-04T18:03:33.190+0000 7f6fa8df1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:03:33.577 INFO:tasks.workunit.client.0.smithi137.stderr: -135> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient(hunting): handle_auth_done global_id 5981 payload 824 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr: -134> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient: _finish_hunting 0 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr: -133> 2024-03-04T18:03:33.190+0000 7f6fa7def640 1 monclient: found mon.noname-c 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr: -132> 2024-03-04T18:03:33.190+0000 7f6fa7def640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.137:3301/0 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr: -131> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr: -130> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr: -129> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: dump: 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.578 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.579 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.579 INFO:tasks.workunit.client.0.smithi137.stderr: -128> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: _finish_auth 0 2024-03-04T18:03:33.579 INFO:tasks.workunit.client.0.smithi137.stderr: -127> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: _check_auth_tickets 2024-03-04T18:03:33.579 INFO:tasks.workunit.client.0.smithi137.stderr: -126> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:03:33.579 INFO:tasks.workunit.client.0.smithi137.stderr: -125> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:03:33.579 INFO:tasks.workunit.client.0.smithi137.stderr: -124> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:03:33.579 INFO:tasks.workunit.client.0.smithi137.stderr: -123> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 10 monclient: dump: 2024-03-04T18:03:33.579 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:03:33.579 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:03:33.579 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.579 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:03:33.579 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:03:33.579 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:03:33.579 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:03:33.580 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:03:33.580 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:03:33.580 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.580 INFO:tasks.workunit.client.0.smithi137.stderr: -122> 2024-03-04T18:03:33.190+0000 7f6faaad8980 5 monclient: authenticate success, global_id 5981 2024-03-04T18:03:33.580 INFO:tasks.workunit.client.0.smithi137.stderr: -121> 2024-03-04T18:03:33.190+0000 7f6e7f7fe640 4 set_mon_vals no callback set 2024-03-04T18:03:33.580 INFO:tasks.workunit.client.0.smithi137.stderr: -120> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient: _renew_subs 2024-03-04T18:03:33.580 INFO:tasks.workunit.client.0.smithi137.stderr: -119> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.137:3301/0 2024-03-04T18:03:33.580 INFO:tasks.workunit.client.0.smithi137.stderr: -118> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient: _renew_subs 2024-03-04T18:03:33.580 INFO:tasks.workunit.client.0.smithi137.stderr: -117> 2024-03-04T18:03:33.190+0000 7f6faaad8980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.137:3301/0 2024-03-04T18:03:33.580 INFO:tasks.workunit.client.0.smithi137.stderr: -116> 2024-03-04T18:03:33.190+0000 7f6faaad8980 1 librados: init done 2024-03-04T18:03:33.580 INFO:tasks.workunit.client.0.smithi137.stderr: -115> 2024-03-04T18:03:33.190+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command cr dump hook 0x560b3b9c4d58 2024-03-04T18:03:33.580 INFO:tasks.workunit.client.0.smithi137.stderr: -114> 2024-03-04T18:03:33.190+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.580 INFO:tasks.workunit.client.0.smithi137.stderr: -113> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:03:33.580 INFO:tasks.workunit.client.0.smithi137.stderr: -112> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -111> 2024-03-04T18:03:33.190+0000 7f6e7dffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -110> 2024-03-04T18:03:33.190+0000 7f6fa85f0640 10 monclient: get_auth_request con 0x7f6e70065940 auth_method 0 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -109> 2024-03-04T18:03:33.190+0000 7f6fa8df1640 10 monclient: get_auth_request con 0x560b3b9ef650 auth_method 0 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -108> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -107> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 realm 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -106> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -105> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -104> 2024-03-04T18:03:33.194+0000 7f6faaad8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -103> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -102> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -101> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -100> 2024-03-04T18:03:33.194+0000 7f6fa85f0640 10 monclient: get_auth_request con 0x560b3b9f3810 auth_method 0 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -99> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -98> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.581 INFO:tasks.workunit.client.0.smithi137.stderr: -97> 2024-03-04T18:03:33.194+0000 7f6fa7def640 10 monclient: get_auth_request con 0x560b3b9f7070 auth_method 0 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -96> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -95> 2024-03-04T18:03:33.194+0000 7f6faaad8980 20 searching for the correct realm 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -94> 2024-03-04T18:03:33.194+0000 7f6fa8df1640 10 monclient: get_auth_request con 0x560b3b9fbdb0 auth_method 0 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -93> 2024-03-04T18:03:33.198+0000 7f6fa85f0640 10 monclient: get_auth_request con 0x7f6fa00a9070 auth_method 0 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -92> 2024-03-04T18:03:33.198+0000 7f6fa7def640 10 monclient: get_auth_request con 0x7f6f98064b30 auth_method 0 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -91> 2024-03-04T18:03:33.202+0000 7f6fa8df1640 10 monclient: get_auth_request con 0x7f6fa00c9400 auth_method 0 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -90> 2024-03-04T18:03:33.202+0000 7f6fa8df1640 0 -- 172.21.15.137:0/2758601501 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f6fa00c9400 msgr2=0x7f6fa00c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -89> 2024-03-04T18:03:33.394+0000 7f6fa85f0640 10 monclient: get_auth_request con 0x7f6e70065940 auth_method 0 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -88> 2024-03-04T18:03:33.406+0000 7f6fa8df1640 10 monclient: get_auth_request con 0x7f6fa00c9400 auth_method 0 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -87> 2024-03-04T18:03:33.410+0000 7f6fa85f0640 10 monclient: get_auth_request con 0x7f6f9806c360 auth_method 0 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -86> 2024-03-04T18:03:33.414+0000 7f6fa7def640 0 -- 172.21.15.137:0/2758601501 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x560b3b9f7070 msgr2=0x560b3b9f9480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -85> 2024-03-04T18:03:33.414+0000 7f6e7dffb640 1 client.5981.objecter ms_handle_reset 0x560b3b9f7070 session 0x560b3b9f2ba0 osd.3 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -84> 2024-03-04T18:03:33.414+0000 7f6e7dffb640 10 monclient: _renew_subs 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -83> 2024-03-04T18:03:33.414+0000 7f6e7dffb640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.137:3301/0 2024-03-04T18:03:33.582 INFO:tasks.workunit.client.0.smithi137.stderr: -82> 2024-03-04T18:03:33.418+0000 7f6fa7def640 10 monclient: get_auth_request con 0x7f6e70075460 auth_method 0 2024-03-04T18:03:33.583 INFO:tasks.workunit.client.0.smithi137.stderr: -81> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:33.583 INFO:tasks.workunit.client.0.smithi137.stderr: -80> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:33.583 INFO:tasks.workunit.client.0.smithi137.stderr: -79> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:33.583 INFO:tasks.workunit.client.0.smithi137.stderr: -78> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:33.583 INFO:tasks.workunit.client.0.smithi137.stderr: -77> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:33.583 INFO:tasks.workunit.client.0.smithi137.stderr: -76> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:33.583 INFO:tasks.workunit.client.0.smithi137.stderr: -75> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.583 INFO:tasks.workunit.client.0.smithi137.stderr: -74> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.583 INFO:tasks.workunit.client.0.smithi137.stderr: -73> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.583 INFO:tasks.workunit.client.0.smithi137.stderr: -72> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:33.583 INFO:tasks.workunit.client.0.smithi137.stderr: -71> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.583 INFO:tasks.workunit.client.0.smithi137.stderr: -70> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:33.583 INFO:tasks.workunit.client.0.smithi137.stderr: -69> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 zone default found 2024-03-04T18:03:33.583 INFO:tasks.workunit.client.0.smithi137.stderr: -68> 2024-03-04T18:03:33.430+0000 7f6faaad8980 4 Realm: () 2024-03-04T18:03:33.584 INFO:tasks.workunit.client.0.smithi137.stderr: -67> 2024-03-04T18:03:33.430+0000 7f6faaad8980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:33.584 INFO:tasks.workunit.client.0.smithi137.stderr: -66> 2024-03-04T18:03:33.430+0000 7f6faaad8980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:33.584 INFO:tasks.workunit.client.0.smithi137.stderr: -65> 2024-03-04T18:03:33.430+0000 7f6faaad8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:33.584 INFO:tasks.workunit.client.0.smithi137.stderr: -64> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 zonegroup default 2024-03-04T18:03:33.584 INFO:tasks.workunit.client.0.smithi137.stderr: -63> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.584 INFO:tasks.workunit.client.0.smithi137.stderr: -62> 2024-03-04T18:03:33.430+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.584 INFO:tasks.workunit.client.0.smithi137.stderr: -61> 2024-03-04T18:03:33.434+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.584 INFO:tasks.workunit.client.0.smithi137.stderr: -60> 2024-03-04T18:03:33.434+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.584 INFO:tasks.workunit.client.0.smithi137.stderr: -59> 2024-03-04T18:03:33.434+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.584 INFO:tasks.workunit.client.0.smithi137.stderr: -58> 2024-03-04T18:03:33.434+0000 7f6faaad8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.584 INFO:tasks.workunit.client.0.smithi137.stderr: -57> 2024-03-04T18:03:33.434+0000 7f6faaad8980 20 started sync module instance, tier type = 2024-03-04T18:03:33.584 INFO:tasks.workunit.client.0.smithi137.stderr: -56> 2024-03-04T18:03:33.434+0000 7f6faaad8980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:33.584 INFO:tasks.workunit.client.0.smithi137.stderr: -55> 2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=0 2024-03-04T18:03:33.584 INFO:tasks.workunit.client.0.smithi137.stderr: -54> 2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=1 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -53> 2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=2 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -52> 2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=3 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -51> 2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=4 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -50> 2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=5 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -49> 2024-03-04T18:03:33.454+0000 7f6faaad8980 20 add_watcher() i=6 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -48> 2024-03-04T18:03:33.458+0000 7f6faaad8980 20 add_watcher() i=7 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -47> 2024-03-04T18:03:33.458+0000 7f6faaad8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -46> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding auth protocol: cephx 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -45> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding auth protocol: cephx 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -44> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding auth protocol: cephx 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -43> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: secure 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -42> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: crc 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -41> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: secure 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -40> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: crc 2024-03-04T18:03:33.585 INFO:tasks.workunit.client.0.smithi137.stderr: -39> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: secure 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -38> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: crc 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -37> 2024-03-04T18:03:33.462+0000 7f6e5efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -36> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: crc 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -35> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: secure 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -34> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: crc 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -33> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: secure 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -32> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: crc 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -31> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 AuthRegistry(0x7ffcb04c6100) adding con mode: secure 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -30> 2024-03-04T18:03:33.462+0000 7f6faaad8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -29> 2024-03-04T18:03:33.462+0000 7f6faaad8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -28> 2024-03-04T18:03:33.462+0000 7f6faaad8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -27> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command cache list hook 0x560b3b9c9fe0 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -26> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command cache inspect hook 0x560b3b9c9fe0 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -25> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command cache erase hook 0x560b3b9c9fe0 2024-03-04T18:03:33.586 INFO:tasks.workunit.client.0.smithi137.stderr: -24> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command cache zap hook 0x560b3b9c9fe0 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -23> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 note: GC not initialized 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -22> 2024-03-04T18:03:33.462+0000 7f6e5dfdb640 20 reqs_thread_entry: start 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -21> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command sync trace show hook 0x7f6fa00047f0 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -20> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command sync trace history hook 0x7f6fa00047f0 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -19> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command sync trace active hook 0x7f6fa00047f0 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -18> 2024-03-04T18:03:33.462+0000 7f6faaad8980 5 asok(0x560b3b71f130) register_command sync trace active_short hook 0x7f6fa00047f0 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -17> 2024-03-04T18:03:33.462+0000 7f6faaad8980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -16> 2024-03-04T18:03:33.466+0000 7f6faaad8980 20 Filter name: none 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -15> 2024-03-04T18:03:33.466+0000 7f6dcf7fe640 20 reqs_thread_entry: start 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -14> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -13> 2024-03-04T18:03:33.466+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -12> 2024-03-04T18:03:33.466+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -11> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -10> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -9> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 : miss 2024-03-04T18:03:33.587 INFO:tasks.workunit.client.0.smithi137.stderr: -8> 2024-03-04T18:03:33.466+0000 7f6faaad8980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.588 INFO:tasks.workunit.client.0.smithi137.stderr: -7> 2024-03-04T18:03:33.466+0000 7f6faaad8980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:33.588 INFO:tasks.workunit.client.0.smithi137.stderr: -6> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 info.flags=0x17 2024-03-04T18:03:33.588 INFO:tasks.workunit.client.0.smithi137.stderr: -5> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 to cache LRU end 2024-03-04T18:03:33.588 INFO:tasks.workunit.client.0.smithi137.stderr: -4> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:33.588 INFO:tasks.workunit.client.0.smithi137.stderr: -3> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:33.588 INFO:tasks.workunit.client.0.smithi137.stderr: -2> 2024-03-04T18:03:33.466+0000 7f6faaad8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 2024-03-04T18:03:33.588 INFO:tasks.workunit.client.0.smithi137.stderr: -1> 2024-03-04T18:03:33.470+0000 7f6faaad8980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f6faaad8980 time 2024-03-04T18:03:33.473595+0000 2024-03-04T18:03:33.588 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:03:33.588 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.588 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:03:33.588 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f6fae3add7b] 2024-03-04T18:03:33.588 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f6fae3adf2f] 2024-03-04T18:03:33.588 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560b39462281] 2024-03-04T18:03:33.588 INFO:tasks.workunit.client.0.smithi137.stderr: 4: radosgw-admin(+0x8c3b2a) [0x560b39469b2a] 2024-03-04T18:03:33.589 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x560b3946ec93] 2024-03-04T18:03:33.589 INFO:tasks.workunit.client.0.smithi137.stderr: 6: main() 2024-03-04T18:03:33.589 INFO:tasks.workunit.client.0.smithi137.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6fad218d90] 2024-03-04T18:03:33.589 INFO:tasks.workunit.client.0.smithi137.stderr: 8: __libc_start_main() 2024-03-04T18:03:33.589 INFO:tasks.workunit.client.0.smithi137.stderr: 9: _start() 2024-03-04T18:03:33.589 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.589 INFO:tasks.workunit.client.0.smithi137.stderr: 0> 2024-03-04T18:03:33.474+0000 7f6faaad8980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:03:33.589 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7f6faaad8980 thread_name:radosgw-admin 2024-03-04T18:03:33.589 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.589 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:03:33.589 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f6fad231520] 2024-03-04T18:03:33.589 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:03:33.589 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f6fae3addd5] 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f6fae3adf2f] 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560b39462281] 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr: 8: radosgw-admin(+0x8c3b2a) [0x560b39469b2a] 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x243) [0x560b3946ec93] 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr: 10: main() 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f6fad218d90] 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr: 12: __libc_start_main() 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr: 13: _start() 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr:--- logging levels --- 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 none 2024-03-04T18:03:33.590 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 lockdep 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 context 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 crush 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_balancer 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_locker 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log_expire 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_migrator 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 buffer 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 timer 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 filer 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 striper 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 objecter 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rados 2024-03-04T18:03:33.591 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_mirror 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_replay 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_pwl 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 journaler 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objectcacher 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 client 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 osd 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 optracker 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objclass 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 filestore 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 journal 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 ms 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mon 2024-03-04T18:03:33.592 INFO:tasks.workunit.client.0.smithi137.stderr: 0/10 monc 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 paxos 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 tp 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 auth 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 crypto 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 finisher 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 reserver 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 heartbeatmap 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 perfcounter 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 20/20 rgw 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_sync 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_datacache 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_access 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_flight 2024-03-04T18:03:33.593 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 javaclient 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 asok 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 throttle 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 refs 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 compressor 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluestore 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluefs 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 bdev 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 kstore 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 4/ 5 rocksdb 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 fuse 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 2/ 5 mgr 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mgrc 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 dpdk 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 eventtrace 2024-03-04T18:03:33.594 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 prioritycache 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 test 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephsqlite 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_onode 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_odata 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_omap 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_tm 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_t 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_epm 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_lba 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:03:33.595 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cache 2024-03-04T18:03:33.596 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_journal 2024-03-04T18:03:33.596 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_device 2024-03-04T18:03:33.596 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_backref 2024-03-04T18:03:33.596 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 alienstore 2024-03-04T18:03:33.596 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mclock 2024-03-04T18:03:33.596 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cyanstore 2024-03-04T18:03:33.596 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 ceph_exporter 2024-03-04T18:03:33.596 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 memstore 2024-03-04T18:03:33.596 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 trace 2024-03-04T18:03:33.596 INFO:tasks.workunit.client.0.smithi137.stderr: -2/-2 (syslog threshold) 2024-03-04T18:03:33.596 INFO:tasks.workunit.client.0.smithi137.stderr: 99/99 (stderr threshold) 2024-03-04T18:03:33.596 INFO:tasks.workunit.client.0.smithi137.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:03:33.596 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6dcf7fe640 / http_manager 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6e5dfdb640 / http_manager 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6e5efdd640 / rgw_dt_lg_renew 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6e7dffb640 / ms_dispatch 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6e7f7fe640 / io_context_pool 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6e8dffb640 / ms_dispatch 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6e94e09640 / ms_dispatch 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6e9660c640 / io_context_pool 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6fa7def640 / msgr-worker-2 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6fa85f0640 / msgr-worker-1 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6fa8df1640 / msgr-worker-0 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6fa95f2640 / io_context_pool 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: 7f6faaad8980 / radosgw-admin 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: max_recent 500 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: max_new 1000 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr: log_file /var/lib/ceph/crash/2024-03-04T18:03:33.477201Z_8fc888ea-c36b-4686-8b47-7617f3d25876/log 2024-03-04T18:03:33.597 INFO:tasks.workunit.client.0.smithi137.stderr:--- end dump of recent events --- 2024-03-04T18:03:33.598 INFO:tasks.workunit.client.0.smithi137.stderr:Aborted 2024-03-04T18:03:33.598 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-03-04T18:03:33.599 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:33.599 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:33.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.554+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.558+0000 7f99a0a24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.558+0000 7f99a0a24980 20 realm 2024-03-04T18:03:33.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.558+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.558+0000 7f99a0a24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.558+0000 7f99a0a24980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:33.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.558+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.558+0000 7f99a0a24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.558+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.558+0000 7f99a0a24980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:33.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.558+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.558+0000 7f99a0a24980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:33.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.558+0000 7f99a0a24980 20 searching for the correct realm 2024-03-04T18:03:33.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.566+0000 7f999ed3d640 0 -- 172.21.15.137:0/445308121 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f99980fc2c0 msgr2=0x7f99981211a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:33.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.778+0000 7f999dd3b640 0 -- 172.21.15.137:0/445308121 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x55670a9ffa00 msgr2=0x55670aa01e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:33.793 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:33.793 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:33.793 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:33.793 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:33.793 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:33.793 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:33.793 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:33.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.795 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:33.795 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 zone default found 2024-03-04T18:03:33.795 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 4 Realm: () 2024-03-04T18:03:33.795 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:33.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:33.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:33.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 zonegroup default 2024-03-04T18:03:33.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.794+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.798+0000 7f99a0a24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.798+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.797 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.798+0000 7f99a0a24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.797 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.798+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.798 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.798+0000 7f99a0a24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.798 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.798+0000 7f99a0a24980 20 started sync module instance, tier type = 2024-03-04T18:03:33.798 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.798+0000 7f99a0a24980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:33.821 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.822+0000 7f99a0a24980 20 add_watcher() i=0 2024-03-04T18:03:33.821 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.822+0000 7f99a0a24980 20 add_watcher() i=1 2024-03-04T18:03:33.821 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.822+0000 7f99a0a24980 20 add_watcher() i=2 2024-03-04T18:03:33.821 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.822+0000 7f99a0a24980 20 add_watcher() i=3 2024-03-04T18:03:33.821 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.822+0000 7f99a0a24980 20 add_watcher() i=4 2024-03-04T18:03:33.821 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.822+0000 7f99a0a24980 20 add_watcher() i=5 2024-03-04T18:03:33.821 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.822+0000 7f99a0a24980 20 add_watcher() i=6 2024-03-04T18:03:33.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.822+0000 7f99a0a24980 20 add_watcher() i=7 2024-03-04T18:03:33.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.822+0000 7f99a0a24980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:33.828 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.830+0000 7f9854fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:33.829 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.830+0000 7f99a0a24980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:33.829 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.830+0000 7f99a0a24980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:33.829 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.830+0000 7f99a0a24980 5 note: GC not initialized 2024-03-04T18:03:33.830 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.830+0000 7f97cf7fe640 20 reqs_thread_entry: start 2024-03-04T18:03:33.830 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.830+0000 7f99a0a24980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:33.831 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.830+0000 7f99a0a24980 20 Filter name: none 2024-03-04T18:03:33.831 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.830+0000 7f97cd7fa640 20 reqs_thread_entry: start 2024-03-04T18:03:33.831 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.830+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.831 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.830+0000 7f99a0a24980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:33.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.830+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.834+0000 7f99a0a24980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:33.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.834+0000 7f99a0a24980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:33.833 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.834+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.833 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.834+0000 7f99a0a24980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:33.833 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.834+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.833 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.834+0000 7f99a0a24980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:33.834 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.834+0000 7f99a0a24980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:03:33.834 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.834+0000 7f99a0a24980 20 rados->read ofs=0 len=0 2024-03-04T18:03:33.834 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.834+0000 7f99a0a24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:33.836 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.838+0000 7f9854fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:33.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.842+0000 7f99a0a24980 20 remove_watcher() i=0 2024-03-04T18:03:33.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.842+0000 7f99a0a24980 2 removed watcher, disabling cache 2024-03-04T18:03:33.843 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.842+0000 7f99a0a24980 20 remove_watcher() i=1 2024-03-04T18:03:33.845 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.846+0000 7f99a0a24980 20 remove_watcher() i=2 2024-03-04T18:03:33.847 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.846+0000 7f99a0a24980 20 remove_watcher() i=3 2024-03-04T18:03:33.849 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.850+0000 7f99a0a24980 20 remove_watcher() i=4 2024-03-04T18:03:33.852 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.850+0000 7f99a0a24980 20 remove_watcher() i=5 2024-03-04T18:03:33.855 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.854+0000 7f99a0a24980 20 remove_watcher() i=6 2024-03-04T18:03:33.857 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:33.858+0000 7f99a0a24980 20 remove_watcher() i=7 2024-03-04T18:03:33.875 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.875 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:03:33.875 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: abort-at-set-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:03:33.875 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:03:33.876 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:33.877 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:33.913 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:03:33.913 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:03:33.981 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:03:34.018 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:03:34.018 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:03:34.060 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.062+0000 7f186baad980 20 rados->read ofs=0 len=0 2024-03-04T18:03:34.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.062+0000 7f186baad980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:34.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.062+0000 7f186baad980 20 realm 2024-03-04T18:03:34.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.062+0000 7f186baad980 20 rados->read ofs=0 len=0 2024-03-04T18:03:34.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.062+0000 7f186baad980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:34.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.062+0000 7f186baad980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:03:34.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.062+0000 7f186baad980 20 rados->read ofs=0 len=0 2024-03-04T18:03:34.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.062+0000 7f186baad980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:34.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.062+0000 7f186baad980 20 rados->read ofs=0 len=0 2024-03-04T18:03:34.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.066+0000 7f186baad980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:34.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.066+0000 7f186baad980 20 rados->read ofs=0 len=0 2024-03-04T18:03:34.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.066+0000 7f186baad980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:03:34.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.066+0000 7f186baad980 20 searching for the correct realm 2024-03-04T18:03:34.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.070+0000 7f1869dc6640 0 -- 172.21.15.137:0/1200520065 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f1864081100 msgr2=0x7f1864056ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:34.285 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.286+0000 7f1869dc6640 0 -- 172.21.15.137:0/1200520065 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x5558839d7e30 msgr2=0x5558839da2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:03:34.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.294+0000 7f186baad980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:03:34.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.294+0000 7f186baad980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:03:34.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.294+0000 7f186baad980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:03:34.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.294+0000 7f186baad980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:03:34.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.294+0000 7f186baad980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:03:34.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.294+0000 7f186baad980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:03:34.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.294+0000 7f186baad980 20 rados->read ofs=0 len=0 2024-03-04T18:03:34.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.294+0000 7f186baad980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:34.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.294+0000 7f186baad980 20 rados->read ofs=0 len=0 2024-03-04T18:03:34.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.294+0000 7f186baad980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:03:34.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.294+0000 7f186baad980 20 rados->read ofs=0 len=0 2024-03-04T18:03:34.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:03:34.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 20 zone default found 2024-03-04T18:03:34.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 4 Realm: () 2024-03-04T18:03:34.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:03:34.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:03:34.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:03:34.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 20 zonegroup default 2024-03-04T18:03:34.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 20 rados->read ofs=0 len=0 2024-03-04T18:03:34.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:34.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 20 rados->read ofs=0 len=0 2024-03-04T18:03:34.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:34.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 20 rados->read ofs=0 len=0 2024-03-04T18:03:34.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:03:34.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 20 started sync module instance, tier type = 2024-03-04T18:03:34.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.298+0000 7f186baad980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:03:34.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.322+0000 7f186baad980 20 add_watcher() i=0 2024-03-04T18:03:34.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.322+0000 7f186baad980 20 add_watcher() i=1 2024-03-04T18:03:34.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.322+0000 7f186baad980 20 add_watcher() i=2 2024-03-04T18:03:34.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.322+0000 7f186baad980 20 add_watcher() i=3 2024-03-04T18:03:34.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.322+0000 7f186baad980 20 add_watcher() i=4 2024-03-04T18:03:34.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.322+0000 7f186baad980 20 add_watcher() i=5 2024-03-04T18:03:34.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.322+0000 7f186baad980 20 add_watcher() i=6 2024-03-04T18:03:34.324 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.326+0000 7f186baad980 20 add_watcher() i=7 2024-03-04T18:03:34.324 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.326+0000 7f186baad980 2 all 8 watchers are set, enabling cache 2024-03-04T18:03:34.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f171ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:34.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f186baad980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:03:34.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f186baad980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:03:34.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f186baad980 5 note: GC not initialized 2024-03-04T18:03:34.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f171efe5640 20 reqs_thread_entry: start 2024-03-04T18:03:34.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f186baad980 20 init_complete bucket index max shards: 1 2024-03-04T18:03:34.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f186baad980 20 Filter name: none 2024-03-04T18:03:34.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f171cfe1640 20 reqs_thread_entry: start 2024-03-04T18:03:34.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f186baad980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-03-04T18:03:34.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f186baad980 20 rados->read ofs=0 len=0 2024-03-04T18:03:34.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f186baad980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:03:34.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f186baad980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-03-04T18:03:34.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f186baad980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-03-04T18:03:34.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f186baad980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 : miss 2024-03-04T18:03:34.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.330+0000 7f186baad980 20 rados->read ofs=0 len=0 2024-03-04T18:03:34.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.334+0000 7f186baad980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:03:34.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.334+0000 7f186baad980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 info.flags=0x17 2024-03-04T18:03:34.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.334+0000 7f186baad980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 to cache LRU end 2024-03-04T18:03:34.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.334+0000 7f186baad980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:03:34.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.334+0000 7f186baad980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:03:34.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.334+0000 7f186baad980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 2024-03-04T18:03:34.334 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.334+0000 7f186baad980 0 INFO: RGWReshardLock::lock found lock on abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 to be held by another RGW process; skipping for now 2024-03-04T18:03:34.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.334+0000 7f171ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:03:34.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.342+0000 7f186baad980 20 remove_watcher() i=0 2024-03-04T18:03:34.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.342+0000 7f186baad980 2 removed watcher, disabling cache 2024-03-04T18:03:34.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.342+0000 7f186baad980 20 remove_watcher() i=1 2024-03-04T18:03:34.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.346+0000 7f186baad980 20 remove_watcher() i=2 2024-03-04T18:03:34.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.346+0000 7f186baad980 20 remove_watcher() i=3 2024-03-04T18:03:34.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.350+0000 7f186baad980 20 remove_watcher() i=4 2024-03-04T18:03:34.377 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.350+0000 7f186baad980 20 remove_watcher() i=5 2024-03-04T18:03:34.377 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.354+0000 7f186baad980 20 remove_watcher() i=6 2024-03-04T18:03:34.377 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:03:34.358+0000 7f186baad980 20 remove_watcher() i=7 2024-03-04T18:03:34.379 INFO:tasks.workunit.client.0.smithi137.stderr:waiting 30 seconds for reshard lock to expire... 2024-03-04T18:04:04.412 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:04:04.447 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:04.447 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:04.494 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.493+0000 7fbb8b9ce980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.497+0000 7fbb8b9ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.497+0000 7fbb8b9ce980 20 realm 2024-03-04T18:04:04.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.497+0000 7fbb8b9ce980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.497+0000 7fbb8b9ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.497 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.497+0000 7fbb8b9ce980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:04.497 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.497+0000 7fbb8b9ce980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.497 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.497+0000 7fbb8b9ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.497 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.497+0000 7fbb8b9ce980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.497 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.497+0000 7fbb8b9ce980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:04.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.497+0000 7fbb8b9ce980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.497+0000 7fbb8b9ce980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:04.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.497+0000 7fbb8b9ce980 20 searching for the correct realm 2024-03-04T18:04:04.504 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.505+0000 7fbb89ce7640 0 -- 172.21.15.137:0/2591387214 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fbb840103e0 msgr2=0x7fbb84010860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:04.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.721+0000 7fbb89ce7640 0 -- 172.21.15.137:0/2591387214 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x563e0ece49f0 msgr2=0x563e0ece6eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:04.731 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.729+0000 7fbb8b9ce980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.729+0000 7fbb8b9ce980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.729+0000 7fbb8b9ce980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.729+0000 7fbb8b9ce980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.729+0000 7fbb8b9ce980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.729+0000 7fbb8b9ce980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.729+0000 7fbb8b9ce980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.729+0000 7fbb8b9ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.732 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.729+0000 7fbb8b9ce980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:04.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:04.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 20 zone default found 2024-03-04T18:04:04.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 4 Realm: () 2024-03-04T18:04:04.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:04.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:04.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:04.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 20 zonegroup default 2024-03-04T18:04:04.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.735 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.735 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.736 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.733+0000 7fbb8b9ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.736 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.737+0000 7fbb8b9ce980 20 started sync module instance, tier type = 2024-03-04T18:04:04.736 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.737+0000 7fbb8b9ce980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:04.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.757+0000 7fbb8b9ce980 20 add_watcher() i=0 2024-03-04T18:04:04.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.757+0000 7fbb8b9ce980 20 add_watcher() i=1 2024-03-04T18:04:04.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.757+0000 7fbb8b9ce980 20 add_watcher() i=2 2024-03-04T18:04:04.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.757+0000 7fbb8b9ce980 20 add_watcher() i=3 2024-03-04T18:04:04.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.757+0000 7fbb8b9ce980 20 add_watcher() i=4 2024-03-04T18:04:04.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.757+0000 7fbb8b9ce980 20 add_watcher() i=5 2024-03-04T18:04:04.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.757+0000 7fbb8b9ce980 20 add_watcher() i=6 2024-03-04T18:04:04.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.761+0000 7fbb8b9ce980 20 add_watcher() i=7 2024-03-04T18:04:04.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.761+0000 7fbb8b9ce980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:04.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.765+0000 7fba3f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:04.767 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.765+0000 7fbb8b9ce980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:04.767 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.765+0000 7fbb8b9ce980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:04.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.765+0000 7fbb8b9ce980 5 note: GC not initialized 2024-03-04T18:04:04.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fba3e7e4640 20 reqs_thread_entry: start 2024-03-04T18:04:04.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:04.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 20 Filter name: none 2024-03-04T18:04:04.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fb9affff640 20 reqs_thread_entry: start 2024-03-04T18:04:04.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-03-04T18:04:04.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:04:04.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-03-04T18:04:04.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-03-04T18:04:04.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 : miss 2024-03-04T18:04:04.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:04:04.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 info.flags=0x17 2024-03-04T18:04:04.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 to cache LRU end 2024-03-04T18:04:04.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:04.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:04.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.769+0000 7fbb8b9ce980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 2024-03-04T18:04:04.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.777+0000 7fbb8b9ce980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 info.flags=0x17 2024-03-04T18:04:04.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.777+0000 7fbb8b9ce980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 to cache LRU end 2024-03-04T18:04:04.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.777+0000 7fbb8b9ce980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:04.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.777+0000 7fbb8b9ce980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22, ofs0, ns] 2024-03-04T18:04:04.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.777+0000 7fba6cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627864 cookie 94824536383856 notifier 6354 bl.length()=959 2024-03-04T18:04:04.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.777+0000 7fba6cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 info.flags=0x17 2024-03-04T18:04:04.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.777+0000 7fba6cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 to cache LRU end 2024-03-04T18:04:04.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.777+0000 7fba6cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:04.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.781+0000 7fbb8b9ce980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22]): orig_sources= new_sources= 2024-03-04T18:04:04.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.781+0000 7fbb8b9ce980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22]): potential sources added=[] removed=[] 2024-03-04T18:04:04.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.781+0000 7fbb8b9ce980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22]): orig_dests= new_dests= 2024-03-04T18:04:04.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.781+0000 7fbb8b9ce980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22]): potential dests added=[] removed=[] 2024-03-04T18:04:04.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.781+0000 7fbb8b9ce980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22.1.0 2024-03-04T18:04:04.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.781+0000 7fbb8b9ce980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22.1.0 2024-03-04T18:04:04.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.781+0000 7fbb8b9ce980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22.1.1 2024-03-04T18:04:04.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.781+0000 7fbb8b9ce980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22.1.1 2024-03-04T18:04:04.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.781+0000 7fbb8b9ce980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22.0 2024-03-04T18:04:04.787 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.785+0000 7fbb8b9ce980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 : hit (requested=0x17, cached=0x17) 2024-03-04T18:04:04.787 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.785+0000 7fbb8b9ce980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 2024-03-04T18:04:04.789 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.789+0000 7fbb8b9ce980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 info.flags=0x17 2024-03-04T18:04:04.789 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.789+0000 7fbb8b9ce980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 to cache LRU end 2024-03-04T18:04:04.789 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.789+0000 7fbb8b9ce980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:04.789 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.789+0000 7fbb8b9ce980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22, ofs0, ns] 2024-03-04T18:04:04.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.789+0000 7fba5bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627865 cookie 94824536383856 notifier 6354 bl.length()=927 2024-03-04T18:04:04.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.789+0000 7fba5bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 info.flags=0x17 2024-03-04T18:04:04.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.789+0000 7fba5bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 to cache LRU end 2024-03-04T18:04:04.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.789+0000 7fba5bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:04.791 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.789+0000 7fbb8b9ce980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22]): orig_sources= new_sources= 2024-03-04T18:04:04.791 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.789+0000 7fbb8b9ce980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22]): potential sources added=[] removed=[] 2024-03-04T18:04:04.791 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.789+0000 7fbb8b9ce980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22]): orig_dests= new_dests= 2024-03-04T18:04:04.791 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.789+0000 7fbb8b9ce980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22]): potential dests added=[] removed=[] 2024-03-04T18:04:04.793 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.793+0000 7fbb8b9ce980 1 execute INFO: reshard of bucket "abort-at-set-target-layout" completed successfully 2024-03-04T18:04:04.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.793+0000 7fba3f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:04.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.801+0000 7fbb8b9ce980 20 remove_watcher() i=0 2024-03-04T18:04:04.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.801+0000 7fbb8b9ce980 2 removed watcher, disabling cache 2024-03-04T18:04:04.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.801+0000 7fbb8b9ce980 20 remove_watcher() i=1 2024-03-04T18:04:04.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.805+0000 7fbb8b9ce980 20 remove_watcher() i=2 2024-03-04T18:04:04.808 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.805+0000 7fbb8b9ce980 20 remove_watcher() i=3 2024-03-04T18:04:04.808 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.809+0000 7fbb8b9ce980 20 remove_watcher() i=4 2024-03-04T18:04:04.810 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.809+0000 7fbb8b9ce980 20 remove_watcher() i=5 2024-03-04T18:04:04.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.813+0000 7fbb8b9ce980 20 remove_watcher() i=6 2024-03-04T18:04:04.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.813+0000 7fbb8b9ce980 20 remove_watcher() i=7 2024-03-04T18:04:04.831 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-03-04T18:04:04.861 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:04.862 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:04.904 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.905+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.905+0000 7f4050167980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.905+0000 7f4050167980 20 realm 2024-03-04T18:04:04.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.905+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.905+0000 7f4050167980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.905+0000 7f4050167980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:04.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.905+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.905+0000 7f4050167980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.908 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.905+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.908 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.909+0000 7f4050167980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:04.908 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.909+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.909+0000 7f4050167980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:04.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.909+0000 7f4050167980 20 searching for the correct realm 2024-03-04T18:04:04.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.913+0000 7f404e480640 0 -- 172.21.15.137:0/889078981 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f40480148f0 msgr2=0x7f40480153f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:04.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.921+0000 7f404e480640 0 -- 172.21.15.137:0/889078981 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x559e5474e190 msgr2=0x559e54750650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:04.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:04.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:04.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:04.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:04.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:04.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:04.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:04.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:04.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 zone default found 2024-03-04T18:04:04.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 4 Realm: () 2024-03-04T18:04:04.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:04.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:04.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:04.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 zonegroup default 2024-03-04T18:04:04.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 started sync module instance, tier type = 2024-03-04T18:04:04.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.929+0000 7f4050167980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:04.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.949+0000 7f4050167980 20 add_watcher() i=0 2024-03-04T18:04:04.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.949+0000 7f4050167980 20 add_watcher() i=1 2024-03-04T18:04:04.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.949+0000 7f4050167980 20 add_watcher() i=2 2024-03-04T18:04:04.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.949+0000 7f4050167980 20 add_watcher() i=3 2024-03-04T18:04:04.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.949+0000 7f4050167980 20 add_watcher() i=4 2024-03-04T18:04:04.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.949+0000 7f4050167980 20 add_watcher() i=5 2024-03-04T18:04:04.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.949+0000 7f4050167980 20 add_watcher() i=6 2024-03-04T18:04:04.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.949+0000 7f4050167980 20 add_watcher() i=7 2024-03-04T18:04:04.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.949+0000 7f4050167980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:04.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.953+0000 7f3eff7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:04.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.953+0000 7f4050167980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:04.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.953+0000 7f4050167980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:04.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.953+0000 7f4050167980 5 note: GC not initialized 2024-03-04T18:04:04.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.953+0000 7f3efe7fc640 20 reqs_thread_entry: start 2024-03-04T18:04:04.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.953+0000 7f4050167980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:04.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.953+0000 7f4050167980 20 Filter name: none 2024-03-04T18:04:04.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.953+0000 7f3e6ffff640 20 reqs_thread_entry: start 2024-03-04T18:04:04.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.953+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.957+0000 7f4050167980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:04:04.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.957+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.957+0000 7f4050167980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:04:04.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.957+0000 7f4050167980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:04.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.957+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.957+0000 7f4050167980 20 rados_obj.operate() r=0 bl.length=218 2024-03-04T18:04:04.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.957+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.957+0000 7f4050167980 20 rados_obj.operate() r=0 bl.length=390 2024-03-04T18:04:04.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.957+0000 7f4050167980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:04.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.957+0000 7f4050167980 20 rados->read ofs=0 len=0 2024-03-04T18:04:04.959 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.957+0000 7f4050167980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:04.960 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.961+0000 7f3eff7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:04.964 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.965+0000 7f4050167980 20 remove_watcher() i=0 2024-03-04T18:04:04.964 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.965+0000 7f4050167980 2 removed watcher, disabling cache 2024-03-04T18:04:04.966 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.965+0000 7f4050167980 20 remove_watcher() i=1 2024-03-04T18:04:04.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.969+0000 7f4050167980 20 remove_watcher() i=2 2024-03-04T18:04:04.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.969+0000 7f4050167980 20 remove_watcher() i=3 2024-03-04T18:04:04.972 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.969+0000 7f4050167980 20 remove_watcher() i=4 2024-03-04T18:04:04.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.973+0000 7f4050167980 20 remove_watcher() i=5 2024-03-04T18:04:04.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.977+0000 7f4050167980 20 remove_watcher() i=6 2024-03-04T18:04:04.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:04.981+0000 7f4050167980 20 remove_watcher() i=7 2024-03-04T18:04:04.993 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:04.994 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:04:04.994 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: abort-at-set-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.22 num_objects: 19 size_kb: 1 num_shards: 2 2024-03-04T18:04:04.994 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:04.995 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:04.995 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:05.015 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:05.015 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:05.381 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:05.382 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:05.565 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: reshard bucket with EIO injected at block_writes 2024-03-04T18:04:05.565 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:05.568 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:05.568 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:05.685 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:05.685 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:05.768 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:05.768 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:05.853 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:05.853 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:05.935 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:05.935 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:06.018 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.018 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:06.101 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.101 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:06.189 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.190 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:06.287 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.287 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:06.368 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.368 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:06.454 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.454 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:06.535 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.535 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:06.620 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.620 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:06.702 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.702 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:06.790 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.790 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:06.869 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.869 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:06.954 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:06.954 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:07.037 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:07.038 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:07.121 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:07.121 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:07.202 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:07.202 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:07.285 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:07.285 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:07.371 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:07.371 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:07.455 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-03-04T18:04:07.486 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:07.486 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:07.530 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.529+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.532 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.533+0000 7f5c43fb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.532 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.533+0000 7f5c43fb5980 20 realm 2024-03-04T18:04:07.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.533+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.533+0000 7f5c43fb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.533+0000 7f5c43fb5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:07.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.533+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.533+0000 7f5c43fb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.533+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.533+0000 7f5c43fb5980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:07.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.533+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.535 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.533+0000 7f5c43fb5980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:07.535 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.533+0000 7f5c43fb5980 20 searching for the correct realm 2024-03-04T18:04:07.538 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.537+0000 7f5c41acd640 0 -- 172.21.15.137:0/3703209378 >> [v2:172.21.15.137:6808/3491386063,v1:172.21.15.137:6810/3491386063] conn(0x55b924ce4770 msgr2=0x7f5c3c0850b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:07.544 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.545+0000 7f5c41acd640 0 -- 172.21.15.137:0/3703209378 >> [v2:172.21.15.137:6824/1039931256,v1:172.21.15.137:6825/1039931256] conn(0x55b924f21ae0 msgr2=0x55b924f23ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:07.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.553+0000 7f5c43fb5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:07.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.553+0000 7f5c43fb5980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:07.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.553+0000 7f5c43fb5980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:07.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.553+0000 7f5c43fb5980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:07.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.553+0000 7f5c43fb5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:07.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.553+0000 7f5c43fb5980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:07.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.553+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.553+0000 7f5c43fb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.553+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.553+0000 7f5c43fb5980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:07.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.553+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:07.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 20 zone default found 2024-03-04T18:04:07.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 4 Realm: () 2024-03-04T18:04:07.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:07.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:07.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:07.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 20 zonegroup default 2024-03-04T18:04:07.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 20 started sync module instance, tier type = 2024-03-04T18:04:07.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.557+0000 7f5c43fb5980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:07.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.573+0000 7f5c43fb5980 20 add_watcher() i=0 2024-03-04T18:04:07.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.573+0000 7f5c43fb5980 20 add_watcher() i=1 2024-03-04T18:04:07.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.573+0000 7f5c43fb5980 20 add_watcher() i=2 2024-03-04T18:04:07.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.573+0000 7f5c43fb5980 20 add_watcher() i=3 2024-03-04T18:04:07.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.573+0000 7f5c43fb5980 20 add_watcher() i=4 2024-03-04T18:04:07.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.573+0000 7f5c43fb5980 20 add_watcher() i=5 2024-03-04T18:04:07.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.573+0000 7f5c43fb5980 20 add_watcher() i=6 2024-03-04T18:04:07.577 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.577+0000 7f5c43fb5980 20 add_watcher() i=7 2024-03-04T18:04:07.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.577+0000 7f5c43fb5980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:07.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.581+0000 7f5af37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:07.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.581+0000 7f5c43fb5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:07.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.581+0000 7f5c43fb5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:07.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.581+0000 7f5c43fb5980 5 note: GC not initialized 2024-03-04T18:04:07.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.581+0000 7f5af27fc640 20 reqs_thread_entry: start 2024-03-04T18:04:07.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.581+0000 7f5c43fb5980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:07.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.581+0000 7f5c43fb5980 20 Filter name: none 2024-03-04T18:04:07.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.581+0000 7f5a63fff640 20 reqs_thread_entry: start 2024-03-04T18:04:07.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.581+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.581+0000 7f5c43fb5980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:07.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.581+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.585+0000 7f5c43fb5980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:07.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.585+0000 7f5c43fb5980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:07.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.585+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.585+0000 7f5c43fb5980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:07.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.585+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.585+0000 7f5c43fb5980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:07.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.585+0000 7f5c43fb5980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:07.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.585+0000 7f5c43fb5980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.585+0000 7f5c43fb5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.585+0000 7f5af37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:07.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.589+0000 7f5c43fb5980 20 remove_watcher() i=0 2024-03-04T18:04:07.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.589+0000 7f5c43fb5980 2 removed watcher, disabling cache 2024-03-04T18:04:07.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.593+0000 7f5c43fb5980 20 remove_watcher() i=1 2024-03-04T18:04:07.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.597+0000 7f5c43fb5980 20 remove_watcher() i=2 2024-03-04T18:04:07.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.597+0000 7f5c43fb5980 20 remove_watcher() i=3 2024-03-04T18:04:07.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.597+0000 7f5c43fb5980 20 remove_watcher() i=4 2024-03-04T18:04:07.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.601+0000 7f5c43fb5980 20 remove_watcher() i=5 2024-03-04T18:04:07.604 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.605+0000 7f5c43fb5980 20 remove_watcher() i=6 2024-03-04T18:04:07.607 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.605+0000 7f5c43fb5980 20 remove_watcher() i=7 2024-03-04T18:04:07.621 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:07.621 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:04:07.621 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-block-writes id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:04:07.621 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:07.622 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at block_writes 2024-03-04T18:04:07.653 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:07.654 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:07.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.693+0000 7fbe23f15980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.697+0000 7fbe23f15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.697+0000 7fbe23f15980 20 realm 2024-03-04T18:04:07.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.697+0000 7fbe23f15980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.697+0000 7fbe23f15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.697+0000 7fbe23f15980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:07.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.697+0000 7fbe23f15980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.697+0000 7fbe23f15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.697+0000 7fbe23f15980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.697+0000 7fbe23f15980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:07.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.697+0000 7fbe23f15980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.701+0000 7fbe23f15980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:07.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.701+0000 7fbe23f15980 20 searching for the correct realm 2024-03-04T18:04:07.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.705+0000 7fbe2222e640 0 -- 172.21.15.137:0/1002110848 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fbe1c0d3910 msgr2=0x7fbe1c0b63c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:07.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.713+0000 7fbe21a2d640 0 -- 172.21.15.137:0/1002110848 >> [v2:172.21.15.173:6805/879938785,v1:172.21.15.173:6808/879938785] conn(0x7fbe14001720 msgr2=0x7fbe14070f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:07.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.717+0000 7fbe23f15980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:07.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.717+0000 7fbe23f15980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:07.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.717+0000 7fbe23f15980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:07.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.717+0000 7fbe23f15980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:07.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.717+0000 7fbe23f15980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:07.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.717+0000 7fbe23f15980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:07.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.717+0000 7fbe23f15980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:07.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:07.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 zone default found 2024-03-04T18:04:07.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 4 Realm: () 2024-03-04T18:04:07.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:07.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:07.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:07.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 zonegroup default 2024-03-04T18:04:07.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.722 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 started sync module instance, tier type = 2024-03-04T18:04:07.723 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.721+0000 7fbe23f15980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:07.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.745+0000 7fbe23f15980 20 add_watcher() i=0 2024-03-04T18:04:07.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.745+0000 7fbe23f15980 20 add_watcher() i=1 2024-03-04T18:04:07.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.745+0000 7fbe23f15980 20 add_watcher() i=2 2024-03-04T18:04:07.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.745+0000 7fbe23f15980 20 add_watcher() i=3 2024-03-04T18:04:07.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.745+0000 7fbe23f15980 20 add_watcher() i=4 2024-03-04T18:04:07.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.745+0000 7fbe23f15980 20 add_watcher() i=5 2024-03-04T18:04:07.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.745+0000 7fbe23f15980 20 add_watcher() i=6 2024-03-04T18:04:07.749 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.749+0000 7fbe23f15980 20 add_watcher() i=7 2024-03-04T18:04:07.749 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.749+0000 7fbe23f15980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:07.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbcd37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:07.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:07.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:07.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 5 note: GC not initialized 2024-03-04T18:04:07.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbcd27fc640 20 reqs_thread_entry: start 2024-03-04T18:04:07.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:07.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 20 Filter name: none 2024-03-04T18:04:07.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbc43fff640 20 reqs_thread_entry: start 2024-03-04T18:04:07.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2024-03-04T18:04:07.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:07.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2024-03-04T18:04:07.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2024-03-04T18:04:07.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 : miss 2024-03-04T18:04:07.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:07.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 info.flags=0x17 2024-03-04T18:04:07.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 to cache LRU end 2024-03-04T18:04:07.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:07.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:07.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.753+0000 7fbe23f15980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 2024-03-04T18:04:07.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.761+0000 7fbe23f15980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 info.flags=0x17 2024-03-04T18:04:07.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.761+0000 7fbe23f15980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 to cache LRU end 2024-03-04T18:04:07.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.761+0000 7fbe23f15980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:07.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.761+0000 7fbe23f15980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23, ofs0, ns] 2024-03-04T18:04:07.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.761+0000 7fbd057fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627816 cookie 94919545895952 notifier 6408 bl.length()=949 2024-03-04T18:04:07.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.761+0000 7fbd057fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 info.flags=0x17 2024-03-04T18:04:07.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.761+0000 7fbd057fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 to cache LRU end 2024-03-04T18:04:07.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.761+0000 7fbd057fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:07.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.765+0000 7fbe23f15980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): orig_sources= new_sources= 2024-03-04T18:04:07.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.765+0000 7fbe23f15980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): potential sources added=[] removed=[] 2024-03-04T18:04:07.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.765+0000 7fbe23f15980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): orig_dests= new_dests= 2024-03-04T18:04:07.765 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.765+0000 7fbe23f15980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): potential dests added=[] removed=[] 2024-03-04T18:04:07.765 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.765+0000 7fbe23f15980 -1 Injecting error=-5 at location=block_writes 2024-03-04T18:04:07.765 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.765+0000 7fbe23f15980 0 ERROR: init_reshard failed to pause writes to the current index: (5) Input/output error 2024-03-04T18:04:07.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.765+0000 7fbe23f15980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 : hit (requested=0x17, cached=0x17) 2024-03-04T18:04:07.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.765+0000 7fbe23f15980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 2024-03-04T18:04:07.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.765+0000 7fbe23f15980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 info.flags=0x17 2024-03-04T18:04:07.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.765+0000 7fbe23f15980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 to cache LRU end 2024-03-04T18:04:07.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.765+0000 7fbe23f15980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:07.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.765+0000 7fbe23f15980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23, ofs0, ns] 2024-03-04T18:04:07.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.769+0000 7fbd04ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627817 cookie 94919545895952 notifier 6408 bl.length()=917 2024-03-04T18:04:07.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.769+0000 7fbd04ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 info.flags=0x17 2024-03-04T18:04:07.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.769+0000 7fbd04ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 to cache LRU end 2024-03-04T18:04:07.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.769+0000 7fbd04ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:07.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.769+0000 7fbe23f15980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): orig_sources= new_sources= 2024-03-04T18:04:07.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.769+0000 7fbe23f15980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): potential sources added=[] removed=[] 2024-03-04T18:04:07.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.769+0000 7fbe23f15980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): orig_dests= new_dests= 2024-03-04T18:04:07.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.769+0000 7fbe23f15980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): potential dests added=[] removed=[] 2024-03-04T18:04:07.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.773+0000 7fbcd37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:07.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.777+0000 7fbe23f15980 20 remove_watcher() i=0 2024-03-04T18:04:07.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.777+0000 7fbe23f15980 2 removed watcher, disabling cache 2024-03-04T18:04:07.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.781+0000 7fbe23f15980 20 remove_watcher() i=1 2024-03-04T18:04:07.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.781+0000 7fbe23f15980 20 remove_watcher() i=2 2024-03-04T18:04:07.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.785+0000 7fbe23f15980 20 remove_watcher() i=3 2024-03-04T18:04:07.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.785+0000 7fbe23f15980 20 remove_watcher() i=4 2024-03-04T18:04:07.788 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.789+0000 7fbe23f15980 20 remove_watcher() i=5 2024-03-04T18:04:07.791 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.789+0000 7fbe23f15980 20 remove_watcher() i=6 2024-03-04T18:04:07.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.793+0000 7fbe23f15980 20 remove_watcher() i=7 2024-03-04T18:04:07.808 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-03-04T18:04:07.837 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:07.837 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:07.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.877+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.880 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.881+0000 7f2b15941980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.880 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.881+0000 7f2b15941980 20 realm 2024-03-04T18:04:07.880 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.881+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.881+0000 7f2b15941980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.881+0000 7f2b15941980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:07.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.881+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.881+0000 7f2b15941980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:07.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.881+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.881+0000 7f2b15941980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:07.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.881+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:07.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.881+0000 7f2b15941980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:07.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.881+0000 7f2b15941980 20 searching for the correct realm 2024-03-04T18:04:07.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:07.889+0000 7f2b13c5a640 0 -- 172.21.15.137:0/3390154779 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f2b0c035400 msgr2=0x7f2b0c035880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:08.105 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.105+0000 7f2b13c5a640 0 -- 172.21.15.137:0/3390154779 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55a129cdf6d0 msgr2=0x55a129ce1b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:08.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.113+0000 7f2b15941980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:08.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.113+0000 7f2b15941980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:08.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.113+0000 7f2b15941980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:08.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.113+0000 7f2b15941980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:08.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.113+0000 7f2b15941980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:08.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.113+0000 7f2b15941980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:08.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.113+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.113+0000 7f2b15941980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.113+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:08.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:08.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 20 zone default found 2024-03-04T18:04:08.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 4 Realm: () 2024-03-04T18:04:08.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:08.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:08.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:08.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 20 zonegroup default 2024-03-04T18:04:08.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 20 started sync module instance, tier type = 2024-03-04T18:04:08.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.117+0000 7f2b15941980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:08.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.141+0000 7f2b15941980 20 add_watcher() i=0 2024-03-04T18:04:08.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.141+0000 7f2b15941980 20 add_watcher() i=1 2024-03-04T18:04:08.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.141+0000 7f2b15941980 20 add_watcher() i=2 2024-03-04T18:04:08.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.141+0000 7f2b15941980 20 add_watcher() i=3 2024-03-04T18:04:08.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.141+0000 7f2b15941980 20 add_watcher() i=4 2024-03-04T18:04:08.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.141+0000 7f2b15941980 20 add_watcher() i=5 2024-03-04T18:04:08.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.141+0000 7f2b15941980 20 add_watcher() i=6 2024-03-04T18:04:08.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.145+0000 7f2b15941980 20 add_watcher() i=7 2024-03-04T18:04:08.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.145+0000 7f2b15941980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:08.150 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.149+0000 7f29c9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:08.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.149+0000 7f2b15941980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:08.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.149+0000 7f2b15941980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:08.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.149+0000 7f2b15941980 5 note: GC not initialized 2024-03-04T18:04:08.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.149+0000 7f29c8fe1640 20 reqs_thread_entry: start 2024-03-04T18:04:08.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.149+0000 7f2b15941980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:08.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.149+0000 7f2b15941980 20 Filter name: none 2024-03-04T18:04:08.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.149+0000 7f29427fc640 20 reqs_thread_entry: start 2024-03-04T18:04:08.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.153+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.153+0000 7f2b15941980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:08.152 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.153+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.153+0000 7f2b15941980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:08.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.153+0000 7f2b15941980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:08.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.153+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.153+0000 7f2b15941980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:08.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.153+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.153+0000 7f2b15941980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:08.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.153+0000 7f2b15941980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:08.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.153+0000 7f2b15941980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.156 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.157+0000 7f2b15941980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.157+0000 7f29c9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:08.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.161+0000 7f2b15941980 20 remove_watcher() i=0 2024-03-04T18:04:08.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.161+0000 7f2b15941980 2 removed watcher, disabling cache 2024-03-04T18:04:08.163 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.161+0000 7f2b15941980 20 remove_watcher() i=1 2024-03-04T18:04:08.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.165+0000 7f2b15941980 20 remove_watcher() i=2 2024-03-04T18:04:08.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.169+0000 7f2b15941980 20 remove_watcher() i=3 2024-03-04T18:04:08.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.169+0000 7f2b15941980 20 remove_watcher() i=4 2024-03-04T18:04:08.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.173+0000 7f2b15941980 20 remove_watcher() i=5 2024-03-04T18:04:08.175 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.173+0000 7f2b15941980 20 remove_watcher() i=6 2024-03-04T18:04:08.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.177+0000 7f2b15941980 20 remove_watcher() i=7 2024-03-04T18:04:08.202 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:08.202 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:04:08.202 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-block-writes id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:04:08.202 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:08.204 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:08.204 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:08.239 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:08.239 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:08.313 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:08.350 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:08.392 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.393+0000 7fe280937980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.393+0000 7fe280937980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.393+0000 7fe280937980 20 realm 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.393+0000 7fe280937980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.393+0000 7fe280937980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.393+0000 7fe280937980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.393+0000 7fe280937980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.393+0000 7fe280937980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.393+0000 7fe280937980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.397+0000 7fe280937980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:08.396 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.397+0000 7fe280937980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.397+0000 7fe280937980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:08.397 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.397+0000 7fe280937980 20 searching for the correct realm 2024-03-04T18:04:08.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.401+0000 7fe27dc4e640 0 -- 172.21.15.137:0/2651701259 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7fe27003dc20 msgr2=0x7fe270079090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:08.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.613+0000 7fe27dc4e640 0 -- 172.21.15.137:0/2651701259 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x5567638c7be0 msgr2=0x5567638c9fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:08.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.629+0000 7fe280937980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:08.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.629+0000 7fe280937980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:08.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.629+0000 7fe280937980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:08.630 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.629+0000 7fe280937980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:08.630 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.629+0000 7fe280937980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:08.630 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.629+0000 7fe280937980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:08.630 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.629+0000 7fe280937980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.630 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.629+0000 7fe280937980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.630 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.629+0000 7fe280937980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.629+0000 7fe280937980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:08.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.629+0000 7fe280937980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.632 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:08.632 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 20 zone default found 2024-03-04T18:04:08.632 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 4 Realm: () 2024-03-04T18:04:08.632 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:08.632 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:08.632 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:08.632 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 20 zonegroup default 2024-03-04T18:04:08.633 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.633 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.633 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.633 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.633 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 20 started sync module instance, tier type = 2024-03-04T18:04:08.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.633+0000 7fe280937980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:08.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.657+0000 7fe280937980 20 add_watcher() i=0 2024-03-04T18:04:08.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.657+0000 7fe280937980 20 add_watcher() i=1 2024-03-04T18:04:08.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.657+0000 7fe280937980 20 add_watcher() i=2 2024-03-04T18:04:08.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.657+0000 7fe280937980 20 add_watcher() i=3 2024-03-04T18:04:08.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.657+0000 7fe280937980 20 add_watcher() i=4 2024-03-04T18:04:08.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.657+0000 7fe280937980 20 add_watcher() i=5 2024-03-04T18:04:08.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.657+0000 7fe280937980 20 add_watcher() i=6 2024-03-04T18:04:08.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.657+0000 7fe280937980 20 add_watcher() i=7 2024-03-04T18:04:08.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.657+0000 7fe280937980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:08.663 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.661+0000 7fe134fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:08.664 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.661+0000 7fe280937980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:08.664 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.661+0000 7fe280937980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:08.664 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.661+0000 7fe280937980 5 note: GC not initialized 2024-03-04T18:04:08.664 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe0af7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:08.664 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:08.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 20 Filter name: none 2024-03-04T18:04:08.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe0ad7fa640 20 reqs_thread_entry: start 2024-03-04T18:04:08.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2024-03-04T18:04:08.665 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:08.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2024-03-04T18:04:08.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2024-03-04T18:04:08.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 : miss 2024-03-04T18:04:08.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:08.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 info.flags=0x17 2024-03-04T18:04:08.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 to cache LRU end 2024-03-04T18:04:08.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:08.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:08.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.665+0000 7fe280937980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 2024-03-04T18:04:08.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.673+0000 7fe280937980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 info.flags=0x17 2024-03-04T18:04:08.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.673+0000 7fe280937980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 to cache LRU end 2024-03-04T18:04:08.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.673+0000 7fe280937980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:08.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.673+0000 7fe280937980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23, ofs0, ns] 2024-03-04T18:04:08.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.673+0000 7fe161ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627818 cookie 93902540184496 notifier 6110 bl.length()=949 2024-03-04T18:04:08.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.673+0000 7fe161ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 info.flags=0x17 2024-03-04T18:04:08.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.673+0000 7fe161ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 to cache LRU end 2024-03-04T18:04:08.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.673+0000 7fe161ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:08.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.677+0000 7fe280937980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): orig_sources= new_sources= 2024-03-04T18:04:08.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.677+0000 7fe280937980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): potential sources added=[] removed=[] 2024-03-04T18:04:08.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.677+0000 7fe280937980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): orig_dests= new_dests= 2024-03-04T18:04:08.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.677+0000 7fe280937980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): potential dests added=[] removed=[] 2024-03-04T18:04:08.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.677+0000 7fe280937980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23.1.0 2024-03-04T18:04:08.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.677+0000 7fe280937980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23.1.0 2024-03-04T18:04:08.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.677+0000 7fe280937980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23.1.1 2024-03-04T18:04:08.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.677+0000 7fe280937980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23.1.1 2024-03-04T18:04:08.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.677+0000 7fe280937980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23.0 2024-03-04T18:04:08.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.681+0000 7fe280937980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 : hit (requested=0x17, cached=0x17) 2024-03-04T18:04:08.683 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.681+0000 7fe280937980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 2024-03-04T18:04:08.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.685+0000 7fe280937980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 info.flags=0x17 2024-03-04T18:04:08.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.685+0000 7fe280937980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 to cache LRU end 2024-03-04T18:04:08.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.685+0000 7fe280937980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:08.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.685+0000 7fe280937980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23, ofs0, ns] 2024-03-04T18:04:08.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.685+0000 7fe1617fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627819 cookie 93902540184496 notifier 6110 bl.length()=917 2024-03-04T18:04:08.686 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.685+0000 7fe1617fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 info.flags=0x17 2024-03-04T18:04:08.686 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.685+0000 7fe1617fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 to cache LRU end 2024-03-04T18:04:08.686 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.685+0000 7fe1617fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:08.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.685+0000 7fe280937980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): orig_sources= new_sources= 2024-03-04T18:04:08.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.685+0000 7fe280937980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): potential sources added=[] removed=[] 2024-03-04T18:04:08.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.685+0000 7fe280937980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): orig_dests= new_dests= 2024-03-04T18:04:08.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.685+0000 7fe280937980 20 handle_bi_update(): bucket=:error-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23]): potential dests added=[] removed=[] 2024-03-04T18:04:08.690 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.689+0000 7fe280937980 1 execute INFO: reshard of bucket "error-at-block-writes" completed successfully 2024-03-04T18:04:08.693 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.693+0000 7fe134fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:08.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.697+0000 7fe280937980 20 remove_watcher() i=0 2024-03-04T18:04:08.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.697+0000 7fe280937980 2 removed watcher, disabling cache 2024-03-04T18:04:08.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.697+0000 7fe280937980 20 remove_watcher() i=1 2024-03-04T18:04:08.702 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.701+0000 7fe280937980 20 remove_watcher() i=2 2024-03-04T18:04:08.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.705+0000 7fe280937980 20 remove_watcher() i=3 2024-03-04T18:04:08.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.705+0000 7fe280937980 20 remove_watcher() i=4 2024-03-04T18:04:08.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.709+0000 7fe280937980 20 remove_watcher() i=5 2024-03-04T18:04:08.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.709+0000 7fe280937980 20 remove_watcher() i=6 2024-03-04T18:04:08.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.713+0000 7fe280937980 20 remove_watcher() i=7 2024-03-04T18:04:08.731 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-03-04T18:04:08.760 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:08.760 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:08.801 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.801+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.803 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.801+0000 7fc8a3504980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.803 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.801+0000 7fc8a3504980 20 realm 2024-03-04T18:04:08.803 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.801+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.801+0000 7fc8a3504980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.801+0000 7fc8a3504980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:08.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.801+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.805+0000 7fc8a3504980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:08.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.805+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.805+0000 7fc8a3504980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:08.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.805+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:08.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.805+0000 7fc8a3504980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:08.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.805+0000 7fc8a3504980 20 searching for the correct realm 2024-03-04T18:04:08.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:08.809+0000 7fc8a181d640 0 -- 172.21.15.137:0/3536090087 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fc89c017070 msgr2=0x7fc89c06c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:09.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.025+0000 7fc8a181d640 0 -- 172.21.15.137:0/3536090087 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55f8c744d600 msgr2=0x55f8c744fac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:09.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.033+0000 7fc8a3504980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:09.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.033+0000 7fc8a3504980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:09.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.033+0000 7fc8a3504980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:09.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.033+0000 7fc8a3504980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:09.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.033+0000 7fc8a3504980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:09.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.033+0000 7fc8a3504980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:09.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.033+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:09.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.033+0000 7fc8a3504980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:09.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.033+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:09.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:09.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:09.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:09.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 20 zone default found 2024-03-04T18:04:09.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 4 Realm: () 2024-03-04T18:04:09.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:09.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:09.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:09.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 20 zonegroup default 2024-03-04T18:04:09.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:09.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:09.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:09.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:09.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:09.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.037+0000 7fc8a3504980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:09.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.041+0000 7fc8a3504980 20 started sync module instance, tier type = 2024-03-04T18:04:09.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.041+0000 7fc8a3504980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:09.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.061+0000 7fc8a3504980 20 add_watcher() i=0 2024-03-04T18:04:09.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.061+0000 7fc8a3504980 20 add_watcher() i=1 2024-03-04T18:04:09.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.061+0000 7fc8a3504980 20 add_watcher() i=2 2024-03-04T18:04:09.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.061+0000 7fc8a3504980 20 add_watcher() i=3 2024-03-04T18:04:09.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.061+0000 7fc8a3504980 20 add_watcher() i=4 2024-03-04T18:04:09.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.061+0000 7fc8a3504980 20 add_watcher() i=5 2024-03-04T18:04:09.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.061+0000 7fc8a3504980 20 add_watcher() i=6 2024-03-04T18:04:09.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.065+0000 7fc8a3504980 20 add_watcher() i=7 2024-03-04T18:04:09.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.065+0000 7fc8a3504980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:09.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.069+0000 7fc7577e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:09.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.069+0000 7fc8a3504980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:09.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.069+0000 7fc8a3504980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:09.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.069+0000 7fc8a3504980 5 note: GC not initialized 2024-03-04T18:04:09.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.069+0000 7fc7567e4640 20 reqs_thread_entry: start 2024-03-04T18:04:09.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.069+0000 7fc8a3504980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:09.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.069+0000 7fc8a3504980 20 Filter name: none 2024-03-04T18:04:09.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.069+0000 7fc6c7fff640 20 reqs_thread_entry: start 2024-03-04T18:04:09.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.069+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:09.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.069+0000 7fc8a3504980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:09.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.069+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:09.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.073+0000 7fc8a3504980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:09.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.073+0000 7fc8a3504980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:09.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.073+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:09.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.073+0000 7fc8a3504980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:09.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.073+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:09.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.073+0000 7fc8a3504980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:09.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.073+0000 7fc8a3504980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:09.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.073+0000 7fc8a3504980 20 rados->read ofs=0 len=0 2024-03-04T18:04:09.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.073+0000 7fc8a3504980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:09.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.077+0000 7fc7577e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:09.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.081+0000 7fc8a3504980 20 remove_watcher() i=0 2024-03-04T18:04:09.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.081+0000 7fc8a3504980 2 removed watcher, disabling cache 2024-03-04T18:04:09.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.081+0000 7fc8a3504980 20 remove_watcher() i=1 2024-03-04T18:04:09.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.085+0000 7fc8a3504980 20 remove_watcher() i=2 2024-03-04T18:04:09.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.089+0000 7fc8a3504980 20 remove_watcher() i=3 2024-03-04T18:04:09.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.089+0000 7fc8a3504980 20 remove_watcher() i=4 2024-03-04T18:04:09.093 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.093+0000 7fc8a3504980 20 remove_watcher() i=5 2024-03-04T18:04:09.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.093+0000 7fc8a3504980 20 remove_watcher() i=6 2024-03-04T18:04:09.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:09.097+0000 7fc8a3504980 20 remove_watcher() i=7 2024-03-04T18:04:09.119 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:09.119 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:04:09.119 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-block-writes id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.23 num_objects: 19 size_kb: 1 num_shards: 2 2024-03-04T18:04:09.119 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:09.121 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:09.121 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:09.152 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:09.152 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:09.583 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:09.583 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:09.752 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: reshard bucket with abort at block_writes 2024-03-04T18:04:09.752 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:09.755 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:09.755 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:09.865 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:09.865 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:09.944 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:09.944 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:10.020 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:10.020 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:10.108 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:10.108 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:10.188 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:10.214 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:10.270 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:10.270 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:10.356 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:10.357 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:10.439 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:10.439 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:10.521 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:10.521 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:10.604 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:10.604 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:10.698 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:10.698 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:10.782 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:10.782 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:10.864 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:10.864 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:10.951 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:10.951 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:11.026 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:11.026 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:11.107 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:11.107 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:11.188 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:11.188 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:11.283 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:11.283 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:11.367 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:11.367 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:11.447 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:11.447 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:11.527 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:11.527 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:11.607 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-03-04T18:04:11.639 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:11.639 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:11.682 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.681+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.685+0000 7f185dc12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:11.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.685+0000 7f185dc12980 20 realm 2024-03-04T18:04:11.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.685+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.685+0000 7f185dc12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:11.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.685+0000 7f185dc12980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:11.686 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.685+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.686 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.685+0000 7f185dc12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:11.686 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.685+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.685+0000 7f185dc12980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:11.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.685+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.685+0000 7f185dc12980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:11.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.685+0000 7f185dc12980 20 searching for the correct realm 2024-03-04T18:04:11.692 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.693+0000 7f185bf2b640 0 -- 172.21.15.137:0/3173094896 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f18540fcf60 msgr2=0x7f18540a82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:11.909 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.909+0000 7f185bf2b640 0 -- 172.21.15.137:0/3173094896 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x557e80356fb0 msgr2=0x557e80359440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:11.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:11.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:11.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:11.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:11.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:11.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:11.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:11.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:11.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:11.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 zone default found 2024-03-04T18:04:11.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 4 Realm: () 2024-03-04T18:04:11.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:11.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:11.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:11.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 zonegroup default 2024-03-04T18:04:11.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.917+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.921+0000 7f185dc12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:11.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.921+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.921+0000 7f185dc12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:11.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.921+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.921+0000 7f185dc12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:11.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.921+0000 7f185dc12980 20 started sync module instance, tier type = 2024-03-04T18:04:11.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.921+0000 7f185dc12980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:11.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.945+0000 7f185dc12980 20 add_watcher() i=0 2024-03-04T18:04:11.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.945+0000 7f185dc12980 20 add_watcher() i=1 2024-03-04T18:04:11.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.945+0000 7f185dc12980 20 add_watcher() i=2 2024-03-04T18:04:11.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.945+0000 7f185dc12980 20 add_watcher() i=3 2024-03-04T18:04:11.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.945+0000 7f185dc12980 20 add_watcher() i=4 2024-03-04T18:04:11.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.945+0000 7f185dc12980 20 add_watcher() i=5 2024-03-04T18:04:11.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.945+0000 7f185dc12980 20 add_watcher() i=6 2024-03-04T18:04:11.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.945+0000 7f185dc12980 20 add_watcher() i=7 2024-03-04T18:04:11.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.945+0000 7f185dc12980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:11.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.953+0000 7f1711fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:11.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.953+0000 7f185dc12980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:11.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.953+0000 7f185dc12980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:11.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.953+0000 7f185dc12980 5 note: GC not initialized 2024-03-04T18:04:11.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.953+0000 7f1710fe1640 20 reqs_thread_entry: start 2024-03-04T18:04:11.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.953+0000 7f185dc12980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:11.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.953+0000 7f185dc12980 20 Filter name: none 2024-03-04T18:04:11.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.953+0000 7f168a7fc640 20 reqs_thread_entry: start 2024-03-04T18:04:11.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.953+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.957+0000 7f185dc12980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:11.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.957+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.957+0000 7f185dc12980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:11.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.957+0000 7f185dc12980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:11.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.957+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.957+0000 7f185dc12980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:11.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.957+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.959 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.957+0000 7f185dc12980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:11.959 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.957+0000 7f185dc12980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:11.960 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.961+0000 7f185dc12980 20 rados->read ofs=0 len=0 2024-03-04T18:04:11.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.961+0000 7f185dc12980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:11.962 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.961+0000 7f1711fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:11.966 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.965+0000 7f185dc12980 20 remove_watcher() i=0 2024-03-04T18:04:11.966 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.965+0000 7f185dc12980 2 removed watcher, disabling cache 2024-03-04T18:04:11.969 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.969+0000 7f185dc12980 20 remove_watcher() i=1 2024-03-04T18:04:11.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.969+0000 7f185dc12980 20 remove_watcher() i=2 2024-03-04T18:04:11.972 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.973+0000 7f185dc12980 20 remove_watcher() i=3 2024-03-04T18:04:11.975 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.973+0000 7f185dc12980 20 remove_watcher() i=4 2024-03-04T18:04:11.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.977+0000 7f185dc12980 20 remove_watcher() i=5 2024-03-04T18:04:11.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.977+0000 7f185dc12980 20 remove_watcher() i=6 2024-03-04T18:04:11.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:11.981+0000 7f185dc12980 20 remove_watcher() i=7 2024-03-04T18:04:11.998 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:11.998 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:04:11.998 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: abort-at-block-writes id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:04:11.998 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:11.998 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at block_writes 2024-03-04T18:04:12.031 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:12.031 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:12.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 realm 2024-03-04T18:04:12.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.073+0000 7f53f6ab8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:12.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.077+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:12.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.077+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.077+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:12.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.077+0000 7f53f6ab8980 20 searching for the correct realm 2024-03-04T18:04:12.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.081+0000 7f53f3dcf640 0 -- 172.21.15.137:0/1600796126 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f53e40fa060 msgr2=0x7f53e40a5300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:12.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.085+0000 7f53f3dcf640 0 -- 172.21.15.137:0/1600796126 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x556e05b8c560 msgr2=0x556e05b8e970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:12.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:12.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:12.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:12.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:12.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:12.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:12.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:12.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:12.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 zone default found 2024-03-04T18:04:12.097 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 4 Realm: () 2024-03-04T18:04:12.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:12.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:12.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:12.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 zonegroup default 2024-03-04T18:04:12.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 started sync module instance, tier type = 2024-03-04T18:04:12.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:12.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=0 2024-03-04T18:04:12.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=1 2024-03-04T18:04:12.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=2 2024-03-04T18:04:12.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=3 2024-03-04T18:04:12.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=4 2024-03-04T18:04:12.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=5 2024-03-04T18:04:12.117 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=6 2024-03-04T18:04:12.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=7 2024-03-04T18:04:12.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.117+0000 7f53f6ab8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:12.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.121+0000 7f52ab7de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:12.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:12.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:12.122 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 note: GC not initialized 2024-03-04T18:04:12.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.121+0000 7f52aa7dc640 20 reqs_thread_entry: start 2024-03-04T18:04:12.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:12.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 Filter name: none 2024-03-04T18:04:12.123 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.121+0000 7f52a37fe640 20 reqs_thread_entry: start 2024-03-04T18:04:12.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.121+0000 7f53f6ab8980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-03-04T18:04:12.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.124 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.125+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:12.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-03-04T18:04:12.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-03-04T18:04:12.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 : miss 2024-03-04T18:04:12.125 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.125+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.125+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:12.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:12.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:12.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:12.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:12.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 2024-03-04T18:04:12.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.133+0000 7f53f6ab8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:12.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.133+0000 7f53f6ab8980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:12.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.133+0000 7f53f6ab8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:12.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.133+0000 7f53f6ab8980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24, ofs0, ns] 2024-03-04T18:04:12.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.133+0000 7f52cb7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627868 cookie 93931030799136 notifier 6498 bl.length()=949 2024-03-04T18:04:12.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.133+0000 7f52cb7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:12.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.133+0000 7f52cb7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:12.135 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.133+0000 7f52cb7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:12.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.137+0000 7f53f6ab8980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): orig_sources= new_sources= 2024-03-04T18:04:12.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.137+0000 7f53f6ab8980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): potential sources added=[] removed=[] 2024-03-04T18:04:12.137 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f53f6ab8980 time 2024-03-04T18:04:12.141563+0000 2024-03-04T18:04:12.137 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:12.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.137+0000 7f53f6ab8980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): orig_dests= new_dests= 2024-03-04T18:04:12.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.137+0000 7f53f6ab8980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): potential dests added=[] removed=[] 2024-03-04T18:04:12.138 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:12.138 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f53fa38dd7b] 2024-03-04T18:04:12.138 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f53fa38df2f] 2024-03-04T18:04:12.138 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x556e0364d281] 2024-03-04T18:04:12.139 INFO:tasks.workunit.client.0.smithi137.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x556e03659ced] 2024-03-04T18:04:12.139 INFO:tasks.workunit.client.0.smithi137.stderr: 5: main() 2024-03-04T18:04:12.139 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f53f91f8d90] 2024-03-04T18:04:12.139 INFO:tasks.workunit.client.0.smithi137.stderr: 7: __libc_start_main() 2024-03-04T18:04:12.139 INFO:tasks.workunit.client.0.smithi137.stderr: 8: _start() 2024-03-04T18:04:12.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.137+0000 7f53f6ab8980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f53f6ab8980 time 2024-03-04T18:04:12.141563+0000 2024-03-04T18:04:12.139 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:12.139 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.139 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:12.139 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f53fa38dd7b] 2024-03-04T18:04:12.139 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f53fa38df2f] 2024-03-04T18:04:12.139 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x556e0364d281] 2024-03-04T18:04:12.139 INFO:tasks.workunit.client.0.smithi137.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x556e03659ced] 2024-03-04T18:04:12.140 INFO:tasks.workunit.client.0.smithi137.stderr: 5: main() 2024-03-04T18:04:12.140 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f53f91f8d90] 2024-03-04T18:04:12.140 INFO:tasks.workunit.client.0.smithi137.stderr: 7: __libc_start_main() 2024-03-04T18:04:12.140 INFO:tasks.workunit.client.0.smithi137.stderr: 8: _start() 2024-03-04T18:04:12.140 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.140 INFO:tasks.workunit.client.0.smithi137.stderr:*** Caught signal (Aborted) ** 2024-03-04T18:04:12.140 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7f53f6ab8980 thread_name:radosgw-admin 2024-03-04T18:04:12.156 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:12.156 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f53f9211520] 2024-03-04T18:04:12.156 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:04:12.156 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:04:12.156 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:04:12.157 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f53fa38ddd5] 2024-03-04T18:04:12.157 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f53fa38df2f] 2024-03-04T18:04:12.157 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x556e0364d281] 2024-03-04T18:04:12.157 INFO:tasks.workunit.client.0.smithi137.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x556e03659ced] 2024-03-04T18:04:12.157 INFO:tasks.workunit.client.0.smithi137.stderr: 9: main() 2024-03-04T18:04:12.157 INFO:tasks.workunit.client.0.smithi137.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f53f91f8d90] 2024-03-04T18:04:12.157 INFO:tasks.workunit.client.0.smithi137.stderr: 11: __libc_start_main() 2024-03-04T18:04:12.157 INFO:tasks.workunit.client.0.smithi137.stderr: 12: _start() 2024-03-04T18:04:12.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.137+0000 7f53f6ab8980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:04:12.157 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7f53f6ab8980 thread_name:radosgw-admin 2024-03-04T18:04:12.157 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.157 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:12.157 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f53f9211520] 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f53fa38ddd5] 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f53fa38df2f] 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x556e0364d281] 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x556e03659ced] 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr: 9: main() 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f53f91f8d90] 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr: 11: __libc_start_main() 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr: 12: _start() 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr:--- begin dump of recent events --- 2024-03-04T18:04:12.158 INFO:tasks.workunit.client.0.smithi137.stderr: -499> 2024-03-04T18:04:12.053+0000 7f52e260c640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr: -498> 2024-03-04T18:04:12.053+0000 7f52e260c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr: -497> 2024-03-04T18:04:12.053+0000 7f52e260c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr: -496> 2024-03-04T18:04:12.053+0000 7f52e260c640 10 monclient: dump: 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr: -495> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 monclient: authenticate success, global_id 6486 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr: -494> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:12.159 INFO:tasks.workunit.client.0.smithi137.stderr: -493> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 4 set_mon_vals no callback set 2024-03-04T18:04:12.160 INFO:tasks.workunit.client.0.smithi137.stderr: -492> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: shutdown 2024-03-04T18:04:12.160 INFO:tasks.workunit.client.0.smithi137.stderr: -491> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 asok(0x556e058b3130) unregister_commands rotate-key 2024-03-04T18:04:12.160 INFO:tasks.workunit.client.0.smithi137.stderr: -490> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: build_initial_monmap 2024-03-04T18:04:12.160 INFO:tasks.workunit.client.0.smithi137.stderr: -489> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 1 build_initial for_mkfs: 0 2024-03-04T18:04:12.160 INFO:tasks.workunit.client.0.smithi137.stderr: -488> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: monmap: 2024-03-04T18:04:12.160 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:12.160 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:12.160 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:12.160 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:12.160 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:12.160 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.160 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:12.160 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:12.160 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: -487> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding auth protocol: cephx 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: -486> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding auth protocol: cephx 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: -485> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding auth protocol: cephx 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: -484> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: secure 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: -483> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: crc 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: -482> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: secure 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: -481> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: crc 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: -480> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: secure 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: -479> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: crc 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: -478> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: crc 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: -477> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: secure 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: -476> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: crc 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: -475> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: secure 2024-03-04T18:04:12.161 INFO:tasks.workunit.client.0.smithi137.stderr: -474> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: crc 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -473> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: secure 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -472> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -471> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 1 librados: starting msgr at 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -470> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 1 librados: starting objecter 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -469> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command objecter_requests hook 0x556e0592b500 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -468> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 1 librados: setting wanted keys 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -467> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 1 librados: calling monclient init 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -466> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: init 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -465> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding auth protocol: cephx 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -464> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding auth protocol: cephx 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -463> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding auth protocol: cephx 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -462> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: secure 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -461> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: crc 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -460> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: secure 2024-03-04T18:04:12.162 INFO:tasks.workunit.client.0.smithi137.stderr: -459> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: crc 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -458> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: secure 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -457> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: crc 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -456> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: crc 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -455> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: secure 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -454> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: crc 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -453> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: secure 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -452> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: crc 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -451> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: secure 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -450> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -449> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -448> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command rotate-key hook 0x556e05a9b148 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -447> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -446> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -445> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-b con 0x556e059492e0 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:12.163 INFO:tasks.workunit.client.0.smithi137.stderr: -444> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-a con 0x556e0594aad0 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:12.164 INFO:tasks.workunit.client.0.smithi137.stderr: -443> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-c con 0x556e059e2960 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:12.164 INFO:tasks.workunit.client.0.smithi137.stderr: -442> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.164 INFO:tasks.workunit.client.0.smithi137.stderr: -441> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.164 INFO:tasks.workunit.client.0.smithi137.stderr: -440> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.164 INFO:tasks.workunit.client.0.smithi137.stderr: -439> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:12.164 INFO:tasks.workunit.client.0.smithi137.stderr: -438> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): authenticate will time out at 2485.175025s 2024-03-04T18:04:12.164 INFO:tasks.workunit.client.0.smithi137.stderr: -437> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request con 0x556e0594aad0 auth_method 0 2024-03-04T18:04:12.164 INFO:tasks.workunit.client.0.smithi137.stderr: -436> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.164 INFO:tasks.workunit.client.0.smithi137.stderr: -435> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.164 INFO:tasks.workunit.client.0.smithi137.stderr: -434> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.164 INFO:tasks.workunit.client.0.smithi137.stderr: -433> 2024-03-04T18:04:12.053+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request con 0x556e059e2960 auth_method 0 2024-03-04T18:04:12.164 INFO:tasks.workunit.client.0.smithi137.stderr: -432> 2024-03-04T18:04:12.053+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.164 INFO:tasks.workunit.client.0.smithi137.stderr: -431> 2024-03-04T18:04:12.053+0000 7f53f4dd1640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -430> 2024-03-04T18:04:12.053+0000 7f53f4dd1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -429> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -428> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -427> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -426> 2024-03-04T18:04:12.053+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -425> 2024-03-04T18:04:12.053+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -424> 2024-03-04T18:04:12.053+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -423> 2024-03-04T18:04:12.053+0000 7f53f45d0640 10 monclient(hunting): get_auth_request con 0x556e059492e0 auth_method 0 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -422> 2024-03-04T18:04:12.053+0000 7f53f45d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -421> 2024-03-04T18:04:12.053+0000 7f53f45d0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -420> 2024-03-04T18:04:12.053+0000 7f53f45d0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -419> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_done global_id 6489 payload 824 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -418> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient: _finish_hunting 0 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -417> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 1 monclient: found mon.noname-a 2024-03-04T18:04:12.165 INFO:tasks.workunit.client.0.smithi137.stderr: -416> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr: -415> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr: -414> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr: -413> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: dump: 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr: -412> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: _finish_auth 0 2024-03-04T18:04:12.166 INFO:tasks.workunit.client.0.smithi137.stderr: -411> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: _check_auth_tickets 2024-03-04T18:04:12.167 INFO:tasks.workunit.client.0.smithi137.stderr: -410> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 monclient: authenticate success, global_id 6489 2024-03-04T18:04:12.167 INFO:tasks.workunit.client.0.smithi137.stderr: -409> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: _renew_subs 2024-03-04T18:04:12.167 INFO:tasks.workunit.client.0.smithi137.stderr: -408> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.167 INFO:tasks.workunit.client.0.smithi137.stderr: -407> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:12.167 INFO:tasks.workunit.client.0.smithi137.stderr: -406> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.167 INFO:tasks.workunit.client.0.smithi137.stderr: -405> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:12.167 INFO:tasks.workunit.client.0.smithi137.stderr: -404> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: dump: 2024-03-04T18:04:12.167 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.167 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.167 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.167 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.167 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.167 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.167 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.168 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.168 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.168 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.168 INFO:tasks.workunit.client.0.smithi137.stderr: -403> 2024-03-04T18:04:12.053+0000 7f52e260c640 4 set_mon_vals no callback set 2024-03-04T18:04:12.168 INFO:tasks.workunit.client.0.smithi137.stderr: -402> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: _renew_subs 2024-03-04T18:04:12.168 INFO:tasks.workunit.client.0.smithi137.stderr: -401> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.168 INFO:tasks.workunit.client.0.smithi137.stderr: -400> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 1 librados: init done 2024-03-04T18:04:12.168 INFO:tasks.workunit.client.0.smithi137.stderr: -399> 2024-03-04T18:04:12.057+0000 7f52e0e09640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:04:12.168 INFO:tasks.workunit.client.0.smithi137.stderr: -398> 2024-03-04T18:04:12.057+0000 7f52e0e09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:12.168 INFO:tasks.workunit.client.0.smithi137.stderr: -397> 2024-03-04T18:04:12.057+0000 7f52e0e09640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:12.168 INFO:tasks.workunit.client.0.smithi137.stderr: -396> 2024-03-04T18:04:12.057+0000 7f53f45d0640 10 monclient: get_auth_request con 0x7f52dc065ae0 auth_method 0 2024-03-04T18:04:12.168 INFO:tasks.workunit.client.0.smithi137.stderr: -395> 2024-03-04T18:04:12.057+0000 7f53f4dd1640 10 monclient: get_auth_request con 0x556e05aa65b0 auth_method 0 2024-03-04T18:04:12.168 INFO:tasks.workunit.client.0.smithi137.stderr: -394> 2024-03-04T18:04:12.061+0000 7f53f45d0640 10 monclient: get_auth_request con 0x556e05ae5b10 auth_method 0 2024-03-04T18:04:12.168 INFO:tasks.workunit.client.0.smithi137.stderr: -393> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient: get_auth_request con 0x556e05ae9e70 auth_method 0 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr: -392> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient: build_initial_monmap 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr: -391> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 1 build_initial for_mkfs: 0 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr: -390> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient: monmap: 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr: -389> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding auth protocol: cephx 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr: -388> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding auth protocol: cephx 2024-03-04T18:04:12.169 INFO:tasks.workunit.client.0.smithi137.stderr: -387> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding auth protocol: cephx 2024-03-04T18:04:12.170 INFO:tasks.workunit.client.0.smithi137.stderr: -386> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: secure 2024-03-04T18:04:12.170 INFO:tasks.workunit.client.0.smithi137.stderr: -385> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: crc 2024-03-04T18:04:12.170 INFO:tasks.workunit.client.0.smithi137.stderr: -384> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: secure 2024-03-04T18:04:12.170 INFO:tasks.workunit.client.0.smithi137.stderr: -383> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: crc 2024-03-04T18:04:12.170 INFO:tasks.workunit.client.0.smithi137.stderr: -382> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: secure 2024-03-04T18:04:12.170 INFO:tasks.workunit.client.0.smithi137.stderr: -381> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: crc 2024-03-04T18:04:12.170 INFO:tasks.workunit.client.0.smithi137.stderr: -380> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: crc 2024-03-04T18:04:12.170 INFO:tasks.workunit.client.0.smithi137.stderr: -379> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: secure 2024-03-04T18:04:12.170 INFO:tasks.workunit.client.0.smithi137.stderr: -378> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: crc 2024-03-04T18:04:12.170 INFO:tasks.workunit.client.0.smithi137.stderr: -377> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: secure 2024-03-04T18:04:12.170 INFO:tasks.workunit.client.0.smithi137.stderr: -376> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: crc 2024-03-04T18:04:12.170 INFO:tasks.workunit.client.0.smithi137.stderr: -375> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: secure 2024-03-04T18:04:12.170 INFO:tasks.workunit.client.0.smithi137.stderr: -374> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.170 INFO:tasks.workunit.client.0.smithi137.stderr: -373> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command objecter_requests cmddesc objecter_requests hook 0x556e05af4240 EEXIST 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -372> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient: init 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -371> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding auth protocol: cephx 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -370> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding auth protocol: cephx 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -369> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding auth protocol: cephx 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -368> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: secure 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -367> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: crc 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -366> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: secure 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -365> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: crc 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -364> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: secure 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -363> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: crc 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -362> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: crc 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -361> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: secure 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -360> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: crc 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -359> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: secure 2024-03-04T18:04:12.171 INFO:tasks.workunit.client.0.smithi137.stderr: -358> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: crc 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -357> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: secure 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -356> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -355> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -354> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command rotate-key cmddesc rotate-key hook 0x556e05aed3e0 EEXIST 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -353> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -352> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -351> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-b con 0x556e05b18ea0 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -350> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-a con 0x556e05b19280 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -349> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-c con 0x556e05b14440 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -348> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -347> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -346> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -345> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -344> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): authenticate will time out at 2485.183501s 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -343> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request con 0x556e05b19280 auth_method 0 2024-03-04T18:04:12.172 INFO:tasks.workunit.client.0.smithi137.stderr: -342> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -341> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -340> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -339> 2024-03-04T18:04:12.061+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request con 0x556e05b14440 auth_method 0 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -338> 2024-03-04T18:04:12.061+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -337> 2024-03-04T18:04:12.061+0000 7f53f4dd1640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -336> 2024-03-04T18:04:12.061+0000 7f53f4dd1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -335> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -334> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -333> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -332> 2024-03-04T18:04:12.061+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -331> 2024-03-04T18:04:12.061+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -330> 2024-03-04T18:04:12.061+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -329> 2024-03-04T18:04:12.061+0000 7f53f45d0640 10 monclient(hunting): get_auth_request con 0x556e05b18ea0 auth_method 0 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -328> 2024-03-04T18:04:12.061+0000 7f53f45d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.173 INFO:tasks.workunit.client.0.smithi137.stderr: -327> 2024-03-04T18:04:12.061+0000 7f53f45d0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr: -326> 2024-03-04T18:04:12.061+0000 7f53f45d0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr: -325> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_done global_id 6492 payload 824 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr: -324> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient: _finish_hunting 0 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr: -323> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 1 monclient: found mon.noname-a 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr: -322> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr: -321> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr: -320> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr: -319> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: dump: 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.174 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.175 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.175 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.175 INFO:tasks.workunit.client.0.smithi137.stderr: -318> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: _finish_auth 0 2024-03-04T18:04:12.175 INFO:tasks.workunit.client.0.smithi137.stderr: -317> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: _check_auth_tickets 2024-03-04T18:04:12.175 INFO:tasks.workunit.client.0.smithi137.stderr: -316> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:12.175 INFO:tasks.workunit.client.0.smithi137.stderr: -315> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.175 INFO:tasks.workunit.client.0.smithi137.stderr: -314> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:12.175 INFO:tasks.workunit.client.0.smithi137.stderr: -313> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: dump: 2024-03-04T18:04:12.175 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.175 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.175 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.175 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.175 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr: -312> 2024-03-04T18:04:12.065+0000 7f53f55d2640 4 set_mon_vals no callback set 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr: -311> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 monclient: authenticate success, global_id 6492 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr: -310> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: _renew_subs 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr: -309> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr: -308> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: _renew_subs 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr: -307> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr: -306> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr: -305> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr: -304> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr: -303> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: get_monmap_and_config 2024-03-04T18:04:12.176 INFO:tasks.workunit.client.0.smithi137.stderr: -302> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: build_initial_monmap 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr: -301> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 1 build_initial for_mkfs: 0 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr: -300> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: monmap: 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr: -299> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding auth protocol: cephx 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr: -298> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding auth protocol: cephx 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr: -297> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding auth protocol: cephx 2024-03-04T18:04:12.177 INFO:tasks.workunit.client.0.smithi137.stderr: -296> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -295> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -294> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -293> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -292> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -291> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -290> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -289> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -288> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -287> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -286> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -285> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -284> 2024-03-04T18:04:12.065+0000 7f53f45d0640 10 monclient: get_auth_request con 0x7f52cc065880 auth_method 0 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -283> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -282> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: init 2024-03-04T18:04:12.178 INFO:tasks.workunit.client.0.smithi137.stderr: -281> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding auth protocol: cephx 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -280> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding auth protocol: cephx 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -279> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding auth protocol: cephx 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -278> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: secure 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -277> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: crc 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -276> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: secure 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -275> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: crc 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -274> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: secure 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -273> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: crc 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -272> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: crc 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -271> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: secure 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -270> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: crc 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -269> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: secure 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -268> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: crc 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -267> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: secure 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -266> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.179 INFO:tasks.workunit.client.0.smithi137.stderr: -265> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -264> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command rotate-key cmddesc rotate-key hook 0x7fff0b3db048 EEXIST 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -263> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -262> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -261> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-b con 0x556e05b5cd30 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -260> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-a con 0x556e05b6b190 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -259> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-c con 0x556e05b70110 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -258> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -257> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -256> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -255> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -254> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): authenticate will time out at 2485.187315s 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -253> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request con 0x556e05b6b190 auth_method 0 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -252> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -251> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.180 INFO:tasks.workunit.client.0.smithi137.stderr: -250> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.181 INFO:tasks.workunit.client.0.smithi137.stderr: -249> 2024-03-04T18:04:12.065+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request con 0x556e05b70110 auth_method 0 2024-03-04T18:04:12.181 INFO:tasks.workunit.client.0.smithi137.stderr: -248> 2024-03-04T18:04:12.065+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.181 INFO:tasks.workunit.client.0.smithi137.stderr: -247> 2024-03-04T18:04:12.065+0000 7f53f4dd1640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.181 INFO:tasks.workunit.client.0.smithi137.stderr: -246> 2024-03-04T18:04:12.065+0000 7f53f4dd1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.181 INFO:tasks.workunit.client.0.smithi137.stderr: -245> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.181 INFO:tasks.workunit.client.0.smithi137.stderr: -244> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.181 INFO:tasks.workunit.client.0.smithi137.stderr: -243> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.181 INFO:tasks.workunit.client.0.smithi137.stderr: -242> 2024-03-04T18:04:12.065+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.181 INFO:tasks.workunit.client.0.smithi137.stderr: -241> 2024-03-04T18:04:12.065+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.181 INFO:tasks.workunit.client.0.smithi137.stderr: -240> 2024-03-04T18:04:12.065+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.181 INFO:tasks.workunit.client.0.smithi137.stderr: -239> 2024-03-04T18:04:12.065+0000 7f53f45d0640 10 monclient(hunting): get_auth_request con 0x556e05b5cd30 auth_method 0 2024-03-04T18:04:12.181 INFO:tasks.workunit.client.0.smithi137.stderr: -238> 2024-03-04T18:04:12.065+0000 7f53f45d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.181 INFO:tasks.workunit.client.0.smithi137.stderr: -237> 2024-03-04T18:04:12.065+0000 7f53f45d0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.181 INFO:tasks.workunit.client.0.smithi137.stderr: -236> 2024-03-04T18:04:12.065+0000 7f53f45d0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.182 INFO:tasks.workunit.client.0.smithi137.stderr: -235> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_done global_id 6495 payload 274 2024-03-04T18:04:12.182 INFO:tasks.workunit.client.0.smithi137.stderr: -234> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient: _finish_hunting 0 2024-03-04T18:04:12.182 INFO:tasks.workunit.client.0.smithi137.stderr: -233> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 1 monclient: found mon.noname-a 2024-03-04T18:04:12.182 INFO:tasks.workunit.client.0.smithi137.stderr: -232> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.182 INFO:tasks.workunit.client.0.smithi137.stderr: -231> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.182 INFO:tasks.workunit.client.0.smithi137.stderr: -230> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:12.182 INFO:tasks.workunit.client.0.smithi137.stderr: -229> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: dump: 2024-03-04T18:04:12.182 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.182 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.182 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.182 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.182 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.182 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.182 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr: -228> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: _finish_auth 0 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr: -227> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: _check_auth_tickets 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr: -226> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr: -225> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr: -224> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr: -223> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: dump: 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.183 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.184 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.184 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.184 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.184 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.184 INFO:tasks.workunit.client.0.smithi137.stderr: -222> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 monclient: authenticate success, global_id 6495 2024-03-04T18:04:12.184 INFO:tasks.workunit.client.0.smithi137.stderr: -221> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:12.184 INFO:tasks.workunit.client.0.smithi137.stderr: -220> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 4 set_mon_vals no callback set 2024-03-04T18:04:12.184 INFO:tasks.workunit.client.0.smithi137.stderr: -219> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: shutdown 2024-03-04T18:04:12.184 INFO:tasks.workunit.client.0.smithi137.stderr: -218> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: build_initial_monmap 2024-03-04T18:04:12.184 INFO:tasks.workunit.client.0.smithi137.stderr: -217> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 1 build_initial for_mkfs: 0 2024-03-04T18:04:12.184 INFO:tasks.workunit.client.0.smithi137.stderr: -216> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: monmap: 2024-03-04T18:04:12.184 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:12.184 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:12.184 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr: -215> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding auth protocol: cephx 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr: -214> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding auth protocol: cephx 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr: -213> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding auth protocol: cephx 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr: -212> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr: -211> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr: -210> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr: -209> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr: -208> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.185 INFO:tasks.workunit.client.0.smithi137.stderr: -207> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -206> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -205> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -204> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -203> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -202> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -201> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -200> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -199> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 1 librados: starting msgr at 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -198> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 1 librados: starting objecter 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -197> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command objecter_requests cmddesc objecter_requests hook 0x556e05b65ee0 EEXIST 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -196> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 1 librados: setting wanted keys 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -195> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 1 librados: calling monclient init 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -194> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: init 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -193> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding auth protocol: cephx 2024-03-04T18:04:12.186 INFO:tasks.workunit.client.0.smithi137.stderr: -192> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding auth protocol: cephx 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -191> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding auth protocol: cephx 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -190> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: secure 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -189> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: crc 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -188> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: secure 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -187> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: crc 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -186> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: secure 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -185> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: crc 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -184> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: crc 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -183> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: secure 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -182> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: crc 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -181> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: secure 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -180> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: crc 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -179> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: secure 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -178> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -177> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.187 INFO:tasks.workunit.client.0.smithi137.stderr: -176> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command rotate-key cmddesc rotate-key hook 0x556e05b35d08 EEXIST 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -175> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -174> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: _add_conns ranks=[0,1,2] 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -173> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-a con 0x556e05b5cd30 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -172> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-b con 0x556e05b6b190 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -171> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-c con 0x556e05b70110 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -170> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -169> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -168> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -167> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -166> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): authenticate will time out at 2485.189950s 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -165> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): get_auth_request con 0x556e05b5cd30 auth_method 0 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -164> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -163> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -162> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.188 INFO:tasks.workunit.client.0.smithi137.stderr: -161> 2024-03-04T18:04:12.069+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request con 0x556e05b70110 auth_method 0 2024-03-04T18:04:12.189 INFO:tasks.workunit.client.0.smithi137.stderr: -160> 2024-03-04T18:04:12.069+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.189 INFO:tasks.workunit.client.0.smithi137.stderr: -159> 2024-03-04T18:04:12.069+0000 7f53f4dd1640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.189 INFO:tasks.workunit.client.0.smithi137.stderr: -158> 2024-03-04T18:04:12.069+0000 7f53f4dd1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.189 INFO:tasks.workunit.client.0.smithi137.stderr: -157> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.189 INFO:tasks.workunit.client.0.smithi137.stderr: -156> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.189 INFO:tasks.workunit.client.0.smithi137.stderr: -155> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.189 INFO:tasks.workunit.client.0.smithi137.stderr: -154> 2024-03-04T18:04:12.069+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.189 INFO:tasks.workunit.client.0.smithi137.stderr: -153> 2024-03-04T18:04:12.069+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.189 INFO:tasks.workunit.client.0.smithi137.stderr: -152> 2024-03-04T18:04:12.069+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.189 INFO:tasks.workunit.client.0.smithi137.stderr: -151> 2024-03-04T18:04:12.069+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request con 0x556e05b6b190 auth_method 0 2024-03-04T18:04:12.189 INFO:tasks.workunit.client.0.smithi137.stderr: -150> 2024-03-04T18:04:12.069+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.189 INFO:tasks.workunit.client.0.smithi137.stderr: -149> 2024-03-04T18:04:12.069+0000 7f53f3dcf640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.189 INFO:tasks.workunit.client.0.smithi137.stderr: -148> 2024-03-04T18:04:12.069+0000 7f53f3dcf640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.189 INFO:tasks.workunit.client.0.smithi137.stderr: -147> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): handle_auth_done global_id 6498 payload 824 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr: -146> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient: _finish_hunting 0 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr: -145> 2024-03-04T18:04:12.069+0000 7f53f45d0640 1 monclient: found mon.noname-a 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr: -144> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr: -143> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr: -142> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr: -141> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: dump: 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.190 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr: -140> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: _finish_auth 0 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr: -139> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: _check_auth_tickets 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr: -138> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr: -137> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr: -136> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr: -135> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: dump: 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.191 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.192 INFO:tasks.workunit.client.0.smithi137.stderr: -134> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 monclient: authenticate success, global_id 6498 2024-03-04T18:04:12.192 INFO:tasks.workunit.client.0.smithi137.stderr: -133> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 4 set_mon_vals no callback set 2024-03-04T18:04:12.192 INFO:tasks.workunit.client.0.smithi137.stderr: -132> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: _renew_subs 2024-03-04T18:04:12.192 INFO:tasks.workunit.client.0.smithi137.stderr: -131> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.192 INFO:tasks.workunit.client.0.smithi137.stderr: -130> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: _renew_subs 2024-03-04T18:04:12.192 INFO:tasks.workunit.client.0.smithi137.stderr: -129> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.192 INFO:tasks.workunit.client.0.smithi137.stderr: -128> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 1 librados: init done 2024-03-04T18:04:12.192 INFO:tasks.workunit.client.0.smithi137.stderr: -127> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command cr dump hook 0x556e05b6b6c8 2024-03-04T18:04:12.192 INFO:tasks.workunit.client.0.smithi137.stderr: -126> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.192 INFO:tasks.workunit.client.0.smithi137.stderr: -125> 2024-03-04T18:04:12.073+0000 7f52c9ffb640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:04:12.192 INFO:tasks.workunit.client.0.smithi137.stderr: -124> 2024-03-04T18:04:12.073+0000 7f52c9ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:12.192 INFO:tasks.workunit.client.0.smithi137.stderr: -123> 2024-03-04T18:04:12.073+0000 7f52c9ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:12.192 INFO:tasks.workunit.client.0.smithi137.stderr: -122> 2024-03-04T18:04:12.073+0000 7f53f3dcf640 10 monclient: get_auth_request con 0x7f52bc065940 auth_method 0 2024-03-04T18:04:12.192 INFO:tasks.workunit.client.0.smithi137.stderr: -121> 2024-03-04T18:04:12.073+0000 7f53f4dd1640 10 monclient: get_auth_request con 0x556e05b84ab0 auth_method 0 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -120> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -119> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 realm 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -118> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -117> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -116> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -115> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -114> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -113> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -112> 2024-03-04T18:04:12.073+0000 7f53f45d0640 10 monclient: get_auth_request con 0x556e05b88e40 auth_method 0 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -111> 2024-03-04T18:04:12.077+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -110> 2024-03-04T18:04:12.077+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -109> 2024-03-04T18:04:12.077+0000 7f53f3dcf640 10 monclient: get_auth_request con 0x556e05b8c560 auth_method 0 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -108> 2024-03-04T18:04:12.077+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -107> 2024-03-04T18:04:12.077+0000 7f53f6ab8980 20 searching for the correct realm 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -106> 2024-03-04T18:04:12.077+0000 7f53f4dd1640 10 monclient: get_auth_request con 0x556e05b912a0 auth_method 0 2024-03-04T18:04:12.193 INFO:tasks.workunit.client.0.smithi137.stderr: -105> 2024-03-04T18:04:12.077+0000 7f53f45d0640 10 monclient: get_auth_request con 0x7f53ec020040 auth_method 0 2024-03-04T18:04:12.194 INFO:tasks.workunit.client.0.smithi137.stderr: -104> 2024-03-04T18:04:12.081+0000 7f53f3dcf640 10 monclient: get_auth_request con 0x7f53e40fa060 auth_method 0 2024-03-04T18:04:12.194 INFO:tasks.workunit.client.0.smithi137.stderr: -103> 2024-03-04T18:04:12.081+0000 7f53f3dcf640 0 -- 172.21.15.137:0/1600796126 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f53e40fa060 msgr2=0x7f53e40a5300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:12.194 INFO:tasks.workunit.client.0.smithi137.stderr: -102> 2024-03-04T18:04:12.081+0000 7f52c9ffb640 1 client.6498.objecter ms_handle_reset 0x7f53e40fa060 session 0x7f53e40c7070 osd.0 2024-03-04T18:04:12.194 INFO:tasks.workunit.client.0.smithi137.stderr: -101> 2024-03-04T18:04:12.081+0000 7f52c9ffb640 10 monclient: _renew_subs 2024-03-04T18:04:12.194 INFO:tasks.workunit.client.0.smithi137.stderr: -100> 2024-03-04T18:04:12.081+0000 7f52c9ffb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.194 INFO:tasks.workunit.client.0.smithi137.stderr: -99> 2024-03-04T18:04:12.081+0000 7f53f3dcf640 10 monclient: get_auth_request con 0x7f52bc075460 auth_method 0 2024-03-04T18:04:12.194 INFO:tasks.workunit.client.0.smithi137.stderr: -98> 2024-03-04T18:04:12.081+0000 7f53f4dd1640 10 monclient: get_auth_request con 0x7f53ec01f920 auth_method 0 2024-03-04T18:04:12.194 INFO:tasks.workunit.client.0.smithi137.stderr: -97> 2024-03-04T18:04:12.085+0000 7f53f45d0640 10 monclient: get_auth_request con 0x7f53e40f1020 auth_method 0 2024-03-04T18:04:12.194 INFO:tasks.workunit.client.0.smithi137.stderr: -96> 2024-03-04T18:04:12.085+0000 7f53f3dcf640 0 -- 172.21.15.137:0/1600796126 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x556e05b8c560 msgr2=0x556e05b8e970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:12.194 INFO:tasks.workunit.client.0.smithi137.stderr: -95> 2024-03-04T18:04:12.085+0000 7f52c9ffb640 1 client.6498.objecter ms_handle_reset 0x556e05b8c560 session 0x556e05b886e0 osd.3 2024-03-04T18:04:12.194 INFO:tasks.workunit.client.0.smithi137.stderr: -94> 2024-03-04T18:04:12.085+0000 7f53f3dcf640 10 monclient: get_auth_request con 0x7f53e40fa060 auth_method 0 2024-03-04T18:04:12.194 INFO:tasks.workunit.client.0.smithi137.stderr: -93> 2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:12.194 INFO:tasks.workunit.client.0.smithi137.stderr: -92> 2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:12.194 INFO:tasks.workunit.client.0.smithi137.stderr: -91> 2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -90> 2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -89> 2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -88> 2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -87> 2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -86> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -85> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -84> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -83> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -82> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -81> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 zone default found 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -80> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 4 Realm: () 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -79> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -78> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -77> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:12.195 INFO:tasks.workunit.client.0.smithi137.stderr: -76> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 zonegroup default 2024-03-04T18:04:12.196 INFO:tasks.workunit.client.0.smithi137.stderr: -75> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.196 INFO:tasks.workunit.client.0.smithi137.stderr: -74> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.196 INFO:tasks.workunit.client.0.smithi137.stderr: -73> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.196 INFO:tasks.workunit.client.0.smithi137.stderr: -72> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.196 INFO:tasks.workunit.client.0.smithi137.stderr: -71> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.196 INFO:tasks.workunit.client.0.smithi137.stderr: -70> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.196 INFO:tasks.workunit.client.0.smithi137.stderr: -69> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 started sync module instance, tier type = 2024-03-04T18:04:12.196 INFO:tasks.workunit.client.0.smithi137.stderr: -68> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:12.196 INFO:tasks.workunit.client.0.smithi137.stderr: -67> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=0 2024-03-04T18:04:12.196 INFO:tasks.workunit.client.0.smithi137.stderr: -66> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=1 2024-03-04T18:04:12.196 INFO:tasks.workunit.client.0.smithi137.stderr: -65> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=2 2024-03-04T18:04:12.196 INFO:tasks.workunit.client.0.smithi137.stderr: -64> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=3 2024-03-04T18:04:12.196 INFO:tasks.workunit.client.0.smithi137.stderr: -63> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=4 2024-03-04T18:04:12.196 INFO:tasks.workunit.client.0.smithi137.stderr: -62> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=5 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -61> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=6 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -60> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=7 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -59> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -58> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding auth protocol: cephx 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -57> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding auth protocol: cephx 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -56> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding auth protocol: cephx 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -55> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: secure 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -54> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: crc 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -53> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: secure 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -52> 2024-03-04T18:04:12.121+0000 7f52ab7de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -51> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: crc 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -50> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: secure 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -49> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: crc 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -48> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: crc 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -47> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: secure 2024-03-04T18:04:12.197 INFO:tasks.workunit.client.0.smithi137.stderr: -46> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: crc 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -45> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: secure 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -44> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: crc 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -43> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: secure 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -42> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -41> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -40> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -39> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command cache list hook 0x556e05b607c0 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -38> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command cache inspect hook 0x556e05b607c0 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -37> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command cache erase hook 0x556e05b607c0 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -36> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command cache zap hook 0x556e05b607c0 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -35> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 note: GC not initialized 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -34> 2024-03-04T18:04:12.121+0000 7f52aa7dc640 20 reqs_thread_entry: start 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -33> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command sync trace show hook 0x7f53e4004470 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -32> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command sync trace history hook 0x7f53e4004470 2024-03-04T18:04:12.198 INFO:tasks.workunit.client.0.smithi137.stderr: -31> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command sync trace active hook 0x7f53e4004470 2024-03-04T18:04:12.199 INFO:tasks.workunit.client.0.smithi137.stderr: -30> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command sync trace active_short hook 0x7f53e4004470 2024-03-04T18:04:12.199 INFO:tasks.workunit.client.0.smithi137.stderr: -29> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:12.199 INFO:tasks.workunit.client.0.smithi137.stderr: -28> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 Filter name: none 2024-03-04T18:04:12.199 INFO:tasks.workunit.client.0.smithi137.stderr: -27> 2024-03-04T18:04:12.121+0000 7f52a37fe640 20 reqs_thread_entry: start 2024-03-04T18:04:12.199 INFO:tasks.workunit.client.0.smithi137.stderr: -26> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-03-04T18:04:12.199 INFO:tasks.workunit.client.0.smithi137.stderr: -25> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.199 INFO:tasks.workunit.client.0.smithi137.stderr: -24> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:12.199 INFO:tasks.workunit.client.0.smithi137.stderr: -23> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-03-04T18:04:12.199 INFO:tasks.workunit.client.0.smithi137.stderr: -22> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-03-04T18:04:12.199 INFO:tasks.workunit.client.0.smithi137.stderr: -21> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 : miss 2024-03-04T18:04:12.199 INFO:tasks.workunit.client.0.smithi137.stderr: -20> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.199 INFO:tasks.workunit.client.0.smithi137.stderr: -19> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:12.199 INFO:tasks.workunit.client.0.smithi137.stderr: -18> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:12.199 INFO:tasks.workunit.client.0.smithi137.stderr: -17> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -16> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -15> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -14> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -13> 2024-03-04T18:04:12.133+0000 7f53f6ab8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -12> 2024-03-04T18:04:12.133+0000 7f53f6ab8980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -11> 2024-03-04T18:04:12.133+0000 7f53f6ab8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -10> 2024-03-04T18:04:12.133+0000 7f53f6ab8980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24, ofs0, ns] 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -9> 2024-03-04T18:04:12.133+0000 7f52cb7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627868 cookie 93931030799136 notifier 6498 bl.length()=949 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -8> 2024-03-04T18:04:12.133+0000 7f52cb7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -7> 2024-03-04T18:04:12.133+0000 7f52cb7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -6> 2024-03-04T18:04:12.133+0000 7f52cb7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -5> 2024-03-04T18:04:12.137+0000 7f53f6ab8980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): orig_sources= new_sources= 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -4> 2024-03-04T18:04:12.137+0000 7f53f6ab8980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): potential sources added=[] removed=[] 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -3> 2024-03-04T18:04:12.137+0000 7f53f6ab8980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): orig_dests= new_dests= 2024-03-04T18:04:12.200 INFO:tasks.workunit.client.0.smithi137.stderr: -2> 2024-03-04T18:04:12.137+0000 7f53f6ab8980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): potential dests added=[] removed=[] 2024-03-04T18:04:12.201 INFO:tasks.workunit.client.0.smithi137.stderr: -1> 2024-03-04T18:04:12.137+0000 7f53f6ab8980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f53f6ab8980 time 2024-03-04T18:04:12.141563+0000 2024-03-04T18:04:12.201 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:12.201 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.201 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:12.201 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f53fa38dd7b] 2024-03-04T18:04:12.201 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f53fa38df2f] 2024-03-04T18:04:12.201 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x556e0364d281] 2024-03-04T18:04:12.201 INFO:tasks.workunit.client.0.smithi137.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x556e03659ced] 2024-03-04T18:04:12.201 INFO:tasks.workunit.client.0.smithi137.stderr: 5: main() 2024-03-04T18:04:12.201 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f53f91f8d90] 2024-03-04T18:04:12.201 INFO:tasks.workunit.client.0.smithi137.stderr: 7: __libc_start_main() 2024-03-04T18:04:12.201 INFO:tasks.workunit.client.0.smithi137.stderr: 8: _start() 2024-03-04T18:04:12.201 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: 0> 2024-03-04T18:04:12.137+0000 7f53f6ab8980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7f53f6ab8980 thread_name:radosgw-admin 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f53f9211520] 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f53fa38ddd5] 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f53fa38df2f] 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x556e0364d281] 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x556e03659ced] 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: 9: main() 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f53f91f8d90] 2024-03-04T18:04:12.202 INFO:tasks.workunit.client.0.smithi137.stderr: 11: __libc_start_main() 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr: 12: _start() 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr:--- logging levels --- 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 none 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 lockdep 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 context 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 crush 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_balancer 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_locker 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log_expire 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_migrator 2024-03-04T18:04:12.203 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 buffer 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 timer 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 filer 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 striper 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 objecter 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rados 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_mirror 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_replay 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_pwl 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 journaler 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objectcacher 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 client 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 osd 2024-03-04T18:04:12.204 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 optracker 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objclass 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 filestore 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 journal 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 ms 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mon 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 0/10 monc 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 paxos 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 tp 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 auth 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 crypto 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 finisher 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 reserver 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 heartbeatmap 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 perfcounter 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 20/20 rgw 2024-03-04T18:04:12.205 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_sync 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_datacache 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_access 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_flight 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 javaclient 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 asok 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 throttle 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 refs 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 compressor 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluestore 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluefs 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 bdev 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 kstore 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 4/ 5 rocksdb 2024-03-04T18:04:12.206 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 fuse 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 2/ 5 mgr 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mgrc 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 dpdk 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 eventtrace 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 prioritycache 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 test 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephsqlite 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_onode 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_odata 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_omap 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_tm 2024-03-04T18:04:12.207 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_t 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_epm 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_lba 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cache 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_journal 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_device 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_backref 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 alienstore 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mclock 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cyanstore 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 ceph_exporter 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 memstore 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 trace 2024-03-04T18:04:12.208 INFO:tasks.workunit.client.0.smithi137.stderr: -2/-2 (syslog threshold) 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr: 99/99 (stderr threshold) 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52a37fe640 / http_manager 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52aa7dc640 / http_manager 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52ab7de640 / rgw_dt_lg_renew 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52c9ffb640 / ms_dispatch 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52cb7fe640 / io_context_pool 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52d9ffb640 / ms_dispatch 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52e0e09640 / ms_dispatch 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52e260c640 / io_context_pool 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr: 7f53f3dcf640 / msgr-worker-2 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr: 7f53f45d0640 / msgr-worker-1 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr: 7f53f4dd1640 / msgr-worker-0 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr: 7f53f55d2640 / io_context_pool 2024-03-04T18:04:12.209 INFO:tasks.workunit.client.0.smithi137.stderr: 7f53f6ab8980 / radosgw-admin 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr: max_recent 500 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr: max_new 1000 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr: log_file /var/log/ceph/ceph-client.admin.87678.log 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr:--- end dump of recent events --- 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr:--- begin dump of recent events --- 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr: -499> 2024-03-04T18:04:12.053+0000 7f52e260c640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr: -498> 2024-03-04T18:04:12.053+0000 7f52e260c640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr: -497> 2024-03-04T18:04:12.053+0000 7f52e260c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr: -496> 2024-03-04T18:04:12.053+0000 7f52e260c640 10 monclient: dump: 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.210 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.211 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.211 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.211 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.211 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.211 INFO:tasks.workunit.client.0.smithi137.stderr: -495> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 monclient: authenticate success, global_id 6486 2024-03-04T18:04:12.211 INFO:tasks.workunit.client.0.smithi137.stderr: -494> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:12.211 INFO:tasks.workunit.client.0.smithi137.stderr: -493> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 4 set_mon_vals no callback set 2024-03-04T18:04:12.211 INFO:tasks.workunit.client.0.smithi137.stderr: -492> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: shutdown 2024-03-04T18:04:12.211 INFO:tasks.workunit.client.0.smithi137.stderr: -491> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 asok(0x556e058b3130) unregister_commands rotate-key 2024-03-04T18:04:12.211 INFO:tasks.workunit.client.0.smithi137.stderr: -490> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: build_initial_monmap 2024-03-04T18:04:12.211 INFO:tasks.workunit.client.0.smithi137.stderr: -489> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 1 build_initial for_mkfs: 0 2024-03-04T18:04:12.211 INFO:tasks.workunit.client.0.smithi137.stderr: -488> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: monmap: 2024-03-04T18:04:12.211 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:12.211 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr: -487> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding auth protocol: cephx 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr: -486> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding auth protocol: cephx 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr: -485> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding auth protocol: cephx 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr: -484> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: secure 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr: -483> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: crc 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr: -482> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: secure 2024-03-04T18:04:12.212 INFO:tasks.workunit.client.0.smithi137.stderr: -481> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: crc 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -480> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: secure 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -479> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: crc 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -478> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: crc 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -477> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: secure 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -476> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: crc 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -475> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: secure 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -474> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: crc 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -473> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aa1360) adding con mode: secure 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -472> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -471> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 1 librados: starting msgr at 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -470> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 1 librados: starting objecter 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -469> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command objecter_requests hook 0x556e0592b500 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -468> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 1 librados: setting wanted keys 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -467> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 1 librados: calling monclient init 2024-03-04T18:04:12.213 INFO:tasks.workunit.client.0.smithi137.stderr: -466> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: init 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -465> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding auth protocol: cephx 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -464> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding auth protocol: cephx 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -463> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding auth protocol: cephx 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -462> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: secure 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -461> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: crc 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -460> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: secure 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -459> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: crc 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -458> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: secure 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -457> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: crc 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -456> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: crc 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -455> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: secure 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -454> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: crc 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -453> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: secure 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -452> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: crc 2024-03-04T18:04:12.214 INFO:tasks.workunit.client.0.smithi137.stderr: -451> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 AuthRegistry(0x556e05a9b000) adding con mode: secure 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -450> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -449> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -448> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command rotate-key hook 0x556e05a9b148 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -447> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -446> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -445> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-b con 0x556e059492e0 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -444> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-a con 0x556e0594aad0 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -443> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-c con 0x556e059e2960 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -442> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -441> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -440> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -439> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -438> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient(hunting): authenticate will time out at 2485.175025s 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -437> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request con 0x556e0594aad0 auth_method 0 2024-03-04T18:04:12.215 INFO:tasks.workunit.client.0.smithi137.stderr: -436> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.216 INFO:tasks.workunit.client.0.smithi137.stderr: -435> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.216 INFO:tasks.workunit.client.0.smithi137.stderr: -434> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.216 INFO:tasks.workunit.client.0.smithi137.stderr: -433> 2024-03-04T18:04:12.053+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request con 0x556e059e2960 auth_method 0 2024-03-04T18:04:12.216 INFO:tasks.workunit.client.0.smithi137.stderr: -432> 2024-03-04T18:04:12.053+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.216 INFO:tasks.workunit.client.0.smithi137.stderr: -431> 2024-03-04T18:04:12.053+0000 7f53f4dd1640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.216 INFO:tasks.workunit.client.0.smithi137.stderr: -430> 2024-03-04T18:04:12.053+0000 7f53f4dd1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.216 INFO:tasks.workunit.client.0.smithi137.stderr: -429> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.216 INFO:tasks.workunit.client.0.smithi137.stderr: -428> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.216 INFO:tasks.workunit.client.0.smithi137.stderr: -427> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.216 INFO:tasks.workunit.client.0.smithi137.stderr: -426> 2024-03-04T18:04:12.053+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.216 INFO:tasks.workunit.client.0.smithi137.stderr: -425> 2024-03-04T18:04:12.053+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.216 INFO:tasks.workunit.client.0.smithi137.stderr: -424> 2024-03-04T18:04:12.053+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.216 INFO:tasks.workunit.client.0.smithi137.stderr: -423> 2024-03-04T18:04:12.053+0000 7f53f45d0640 10 monclient(hunting): get_auth_request con 0x556e059492e0 auth_method 0 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr: -422> 2024-03-04T18:04:12.053+0000 7f53f45d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr: -421> 2024-03-04T18:04:12.053+0000 7f53f45d0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr: -420> 2024-03-04T18:04:12.053+0000 7f53f45d0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr: -419> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_done global_id 6489 payload 824 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr: -418> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient: _finish_hunting 0 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr: -417> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 1 monclient: found mon.noname-a 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr: -416> 2024-03-04T18:04:12.053+0000 7f53f3dcf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr: -415> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr: -414> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr: -413> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: dump: 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.217 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr: -412> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: _finish_auth 0 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr: -411> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: _check_auth_tickets 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr: -410> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 5 monclient: authenticate success, global_id 6489 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr: -409> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: _renew_subs 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr: -408> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr: -407> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr: -406> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr: -405> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr: -404> 2024-03-04T18:04:12.053+0000 7f52e0e09640 10 monclient: dump: 2024-03-04T18:04:12.218 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr: -403> 2024-03-04T18:04:12.053+0000 7f52e260c640 4 set_mon_vals no callback set 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr: -402> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: _renew_subs 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr: -401> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr: -400> 2024-03-04T18:04:12.053+0000 7f53f6ab8980 1 librados: init done 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr: -399> 2024-03-04T18:04:12.057+0000 7f52e0e09640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:04:12.219 INFO:tasks.workunit.client.0.smithi137.stderr: -398> 2024-03-04T18:04:12.057+0000 7f52e0e09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr: -397> 2024-03-04T18:04:12.057+0000 7f52e0e09640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr: -396> 2024-03-04T18:04:12.057+0000 7f53f45d0640 10 monclient: get_auth_request con 0x7f52dc065ae0 auth_method 0 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr: -395> 2024-03-04T18:04:12.057+0000 7f53f4dd1640 10 monclient: get_auth_request con 0x556e05aa65b0 auth_method 0 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr: -394> 2024-03-04T18:04:12.061+0000 7f53f45d0640 10 monclient: get_auth_request con 0x556e05ae5b10 auth_method 0 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr: -393> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient: get_auth_request con 0x556e05ae9e70 auth_method 0 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr: -392> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient: build_initial_monmap 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr: -391> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 1 build_initial for_mkfs: 0 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr: -390> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient: monmap: 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.220 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr: -389> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding auth protocol: cephx 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr: -388> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding auth protocol: cephx 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr: -387> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding auth protocol: cephx 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr: -386> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: secure 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr: -385> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: crc 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr: -384> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: secure 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr: -383> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: crc 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr: -382> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: secure 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr: -381> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: crc 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr: -380> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: crc 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr: -379> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: secure 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr: -378> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: crc 2024-03-04T18:04:12.221 INFO:tasks.workunit.client.0.smithi137.stderr: -377> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: secure 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -376> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: crc 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -375> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aeead0) adding con mode: secure 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -374> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -373> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command objecter_requests cmddesc objecter_requests hook 0x556e05af4240 EEXIST 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -372> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient: init 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -371> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding auth protocol: cephx 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -370> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding auth protocol: cephx 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -369> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding auth protocol: cephx 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -368> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: secure 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -367> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: crc 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -366> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: secure 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -365> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: crc 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -364> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: secure 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -363> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: crc 2024-03-04T18:04:12.222 INFO:tasks.workunit.client.0.smithi137.stderr: -362> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: crc 2024-03-04T18:04:12.223 INFO:tasks.workunit.client.0.smithi137.stderr: -361> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: secure 2024-03-04T18:04:12.223 INFO:tasks.workunit.client.0.smithi137.stderr: -360> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: crc 2024-03-04T18:04:12.223 INFO:tasks.workunit.client.0.smithi137.stderr: -359> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: secure 2024-03-04T18:04:12.223 INFO:tasks.workunit.client.0.smithi137.stderr: -358> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: crc 2024-03-04T18:04:12.223 INFO:tasks.workunit.client.0.smithi137.stderr: -357> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 AuthRegistry(0x556e05aed298) adding con mode: secure 2024-03-04T18:04:12.223 INFO:tasks.workunit.client.0.smithi137.stderr: -356> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.223 INFO:tasks.workunit.client.0.smithi137.stderr: -355> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.223 INFO:tasks.workunit.client.0.smithi137.stderr: -354> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command rotate-key cmddesc rotate-key hook 0x556e05aed3e0 EEXIST 2024-03-04T18:04:12.223 INFO:tasks.workunit.client.0.smithi137.stderr: -353> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:12.223 INFO:tasks.workunit.client.0.smithi137.stderr: -352> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:04:12.223 INFO:tasks.workunit.client.0.smithi137.stderr: -351> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-b con 0x556e05b18ea0 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:12.223 INFO:tasks.workunit.client.0.smithi137.stderr: -350> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-a con 0x556e05b19280 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:12.223 INFO:tasks.workunit.client.0.smithi137.stderr: -349> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-c con 0x556e05b14440 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:12.223 INFO:tasks.workunit.client.0.smithi137.stderr: -348> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -347> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -346> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -345> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -344> 2024-03-04T18:04:12.061+0000 7f53f6ab8980 10 monclient(hunting): authenticate will time out at 2485.183501s 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -343> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request con 0x556e05b19280 auth_method 0 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -342> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -341> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -340> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -339> 2024-03-04T18:04:12.061+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request con 0x556e05b14440 auth_method 0 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -338> 2024-03-04T18:04:12.061+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -337> 2024-03-04T18:04:12.061+0000 7f53f4dd1640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -336> 2024-03-04T18:04:12.061+0000 7f53f4dd1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -335> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -334> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.224 INFO:tasks.workunit.client.0.smithi137.stderr: -333> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr: -332> 2024-03-04T18:04:12.061+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr: -331> 2024-03-04T18:04:12.061+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr: -330> 2024-03-04T18:04:12.061+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr: -329> 2024-03-04T18:04:12.061+0000 7f53f45d0640 10 monclient(hunting): get_auth_request con 0x556e05b18ea0 auth_method 0 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr: -328> 2024-03-04T18:04:12.061+0000 7f53f45d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr: -327> 2024-03-04T18:04:12.061+0000 7f53f45d0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr: -326> 2024-03-04T18:04:12.061+0000 7f53f45d0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr: -325> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_done global_id 6492 payload 824 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr: -324> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient: _finish_hunting 0 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr: -323> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 1 monclient: found mon.noname-a 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr: -322> 2024-03-04T18:04:12.061+0000 7f53f3dcf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr: -321> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr: -320> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr: -319> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: dump: 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.225 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr: -318> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: _finish_auth 0 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr: -317> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: _check_auth_tickets 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr: -316> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr: -315> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr: -314> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr: -313> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 10 monclient: dump: 2024-03-04T18:04:12.226 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr: -312> 2024-03-04T18:04:12.065+0000 7f53f55d2640 4 set_mon_vals no callback set 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr: -311> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 monclient: authenticate success, global_id 6492 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr: -310> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: _renew_subs 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr: -309> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr: -308> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: _renew_subs 2024-03-04T18:04:12.227 INFO:tasks.workunit.client.0.smithi137.stderr: -307> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.228 INFO:tasks.workunit.client.0.smithi137.stderr: -306> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:04:12.228 INFO:tasks.workunit.client.0.smithi137.stderr: -305> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:12.228 INFO:tasks.workunit.client.0.smithi137.stderr: -304> 2024-03-04T18:04:12.065+0000 7f52d9ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:12.228 INFO:tasks.workunit.client.0.smithi137.stderr: -303> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: get_monmap_and_config 2024-03-04T18:04:12.228 INFO:tasks.workunit.client.0.smithi137.stderr: -302> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: build_initial_monmap 2024-03-04T18:04:12.228 INFO:tasks.workunit.client.0.smithi137.stderr: -301> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 1 build_initial for_mkfs: 0 2024-03-04T18:04:12.228 INFO:tasks.workunit.client.0.smithi137.stderr: -300> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: monmap: 2024-03-04T18:04:12.228 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:12.228 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:12.228 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:12.228 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:12.228 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:12.228 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.228 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr: -299> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding auth protocol: cephx 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr: -298> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding auth protocol: cephx 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr: -297> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding auth protocol: cephx 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr: -296> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr: -295> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr: -294> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr: -293> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr: -292> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr: -291> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr: -290> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr: -289> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.229 INFO:tasks.workunit.client.0.smithi137.stderr: -288> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -287> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -286> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -285> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -284> 2024-03-04T18:04:12.065+0000 7f53f45d0640 10 monclient: get_auth_request con 0x7f52cc065880 auth_method 0 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -283> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -282> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: init 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -281> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding auth protocol: cephx 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -280> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding auth protocol: cephx 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -279> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding auth protocol: cephx 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -278> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: secure 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -277> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: crc 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -276> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: secure 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -275> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: crc 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -274> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: secure 2024-03-04T18:04:12.230 INFO:tasks.workunit.client.0.smithi137.stderr: -273> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: crc 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -272> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: crc 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -271> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: secure 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -270> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: crc 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -269> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: secure 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -268> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: crc 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -267> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3daf00) adding con mode: secure 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -266> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -265> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -264> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command rotate-key cmddesc rotate-key hook 0x7fff0b3db048 EEXIST 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -263> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -262> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -261> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-b con 0x556e05b5cd30 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -260> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-a con 0x556e05b6b190 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -259> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-c con 0x556e05b70110 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:12.231 INFO:tasks.workunit.client.0.smithi137.stderr: -258> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.232 INFO:tasks.workunit.client.0.smithi137.stderr: -257> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.232 INFO:tasks.workunit.client.0.smithi137.stderr: -256> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.232 INFO:tasks.workunit.client.0.smithi137.stderr: -255> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:12.232 INFO:tasks.workunit.client.0.smithi137.stderr: -254> 2024-03-04T18:04:12.065+0000 7f53f6ab8980 10 monclient(hunting): authenticate will time out at 2485.187315s 2024-03-04T18:04:12.232 INFO:tasks.workunit.client.0.smithi137.stderr: -253> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request con 0x556e05b6b190 auth_method 0 2024-03-04T18:04:12.232 INFO:tasks.workunit.client.0.smithi137.stderr: -252> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.232 INFO:tasks.workunit.client.0.smithi137.stderr: -251> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.232 INFO:tasks.workunit.client.0.smithi137.stderr: -250> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.232 INFO:tasks.workunit.client.0.smithi137.stderr: -249> 2024-03-04T18:04:12.065+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request con 0x556e05b70110 auth_method 0 2024-03-04T18:04:12.232 INFO:tasks.workunit.client.0.smithi137.stderr: -248> 2024-03-04T18:04:12.065+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.232 INFO:tasks.workunit.client.0.smithi137.stderr: -247> 2024-03-04T18:04:12.065+0000 7f53f4dd1640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.232 INFO:tasks.workunit.client.0.smithi137.stderr: -246> 2024-03-04T18:04:12.065+0000 7f53f4dd1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.232 INFO:tasks.workunit.client.0.smithi137.stderr: -245> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.232 INFO:tasks.workunit.client.0.smithi137.stderr: -244> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.233 INFO:tasks.workunit.client.0.smithi137.stderr: -243> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.233 INFO:tasks.workunit.client.0.smithi137.stderr: -242> 2024-03-04T18:04:12.065+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.233 INFO:tasks.workunit.client.0.smithi137.stderr: -241> 2024-03-04T18:04:12.065+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.233 INFO:tasks.workunit.client.0.smithi137.stderr: -240> 2024-03-04T18:04:12.065+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.233 INFO:tasks.workunit.client.0.smithi137.stderr: -239> 2024-03-04T18:04:12.065+0000 7f53f45d0640 10 monclient(hunting): get_auth_request con 0x556e05b5cd30 auth_method 0 2024-03-04T18:04:12.233 INFO:tasks.workunit.client.0.smithi137.stderr: -238> 2024-03-04T18:04:12.065+0000 7f53f45d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.233 INFO:tasks.workunit.client.0.smithi137.stderr: -237> 2024-03-04T18:04:12.065+0000 7f53f45d0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.233 INFO:tasks.workunit.client.0.smithi137.stderr: -236> 2024-03-04T18:04:12.065+0000 7f53f45d0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.233 INFO:tasks.workunit.client.0.smithi137.stderr: -235> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient(hunting): handle_auth_done global_id 6495 payload 274 2024-03-04T18:04:12.233 INFO:tasks.workunit.client.0.smithi137.stderr: -234> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient: _finish_hunting 0 2024-03-04T18:04:12.233 INFO:tasks.workunit.client.0.smithi137.stderr: -233> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 1 monclient: found mon.noname-a 2024-03-04T18:04:12.233 INFO:tasks.workunit.client.0.smithi137.stderr: -232> 2024-03-04T18:04:12.065+0000 7f53f3dcf640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.233 INFO:tasks.workunit.client.0.smithi137.stderr: -231> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.233 INFO:tasks.workunit.client.0.smithi137.stderr: -230> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr: -229> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: dump: 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr: -228> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: _finish_auth 0 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr: -227> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: _check_auth_tickets 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr: -226> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr: -225> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.234 INFO:tasks.workunit.client.0.smithi137.stderr: -224> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr: -223> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 10 monclient: dump: 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr: -222> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 monclient: authenticate success, global_id 6495 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr: -221> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr: -220> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 4 set_mon_vals no callback set 2024-03-04T18:04:12.235 INFO:tasks.workunit.client.0.smithi137.stderr: -219> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: shutdown 2024-03-04T18:04:12.236 INFO:tasks.workunit.client.0.smithi137.stderr: -218> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: build_initial_monmap 2024-03-04T18:04:12.236 INFO:tasks.workunit.client.0.smithi137.stderr: -217> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 1 build_initial for_mkfs: 0 2024-03-04T18:04:12.236 INFO:tasks.workunit.client.0.smithi137.stderr: -216> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: monmap: 2024-03-04T18:04:12.236 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:12.236 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:12.236 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:12.236 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:12.236 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:12.236 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.236 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:12.236 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:12.236 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:12.236 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.236 INFO:tasks.workunit.client.0.smithi137.stderr: -215> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding auth protocol: cephx 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -214> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding auth protocol: cephx 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -213> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding auth protocol: cephx 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -212> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -211> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -210> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -209> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -208> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -207> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -206> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -205> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -204> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -203> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -202> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: crc 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -201> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b369f0) adding con mode: secure 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -200> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.237 INFO:tasks.workunit.client.0.smithi137.stderr: -199> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 1 librados: starting msgr at 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -198> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 1 librados: starting objecter 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -197> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command objecter_requests cmddesc objecter_requests hook 0x556e05b65ee0 EEXIST 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -196> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 1 librados: setting wanted keys 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -195> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 1 librados: calling monclient init 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -194> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: init 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -193> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding auth protocol: cephx 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -192> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding auth protocol: cephx 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -191> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding auth protocol: cephx 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -190> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: secure 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -189> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: crc 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -188> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: secure 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -187> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: crc 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -186> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: secure 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -185> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: crc 2024-03-04T18:04:12.238 INFO:tasks.workunit.client.0.smithi137.stderr: -184> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: crc 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -183> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: secure 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -182> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: crc 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -181> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: secure 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -180> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: crc 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -179> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 AuthRegistry(0x556e05b35bc0) adding con mode: secure 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -178> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -177> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -176> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command rotate-key cmddesc rotate-key hook 0x556e05b35d08 EEXIST 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -175> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -174> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: _add_conns ranks=[0,1,2] 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -173> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-a con 0x556e05b5cd30 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -172> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-b con 0x556e05b6b190 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -171> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): picked mon.noname-c con 0x556e05b70110 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -170> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.239 INFO:tasks.workunit.client.0.smithi137.stderr: -169> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.240 INFO:tasks.workunit.client.0.smithi137.stderr: -168> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:12.240 INFO:tasks.workunit.client.0.smithi137.stderr: -167> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:12.240 INFO:tasks.workunit.client.0.smithi137.stderr: -166> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient(hunting): authenticate will time out at 2485.189950s 2024-03-04T18:04:12.240 INFO:tasks.workunit.client.0.smithi137.stderr: -165> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): get_auth_request con 0x556e05b5cd30 auth_method 0 2024-03-04T18:04:12.240 INFO:tasks.workunit.client.0.smithi137.stderr: -164> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.240 INFO:tasks.workunit.client.0.smithi137.stderr: -163> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.240 INFO:tasks.workunit.client.0.smithi137.stderr: -162> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.240 INFO:tasks.workunit.client.0.smithi137.stderr: -161> 2024-03-04T18:04:12.069+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request con 0x556e05b70110 auth_method 0 2024-03-04T18:04:12.240 INFO:tasks.workunit.client.0.smithi137.stderr: -160> 2024-03-04T18:04:12.069+0000 7f53f4dd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.240 INFO:tasks.workunit.client.0.smithi137.stderr: -159> 2024-03-04T18:04:12.069+0000 7f53f4dd1640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.240 INFO:tasks.workunit.client.0.smithi137.stderr: -158> 2024-03-04T18:04:12.069+0000 7f53f4dd1640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.240 INFO:tasks.workunit.client.0.smithi137.stderr: -157> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.240 INFO:tasks.workunit.client.0.smithi137.stderr: -156> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.240 INFO:tasks.workunit.client.0.smithi137.stderr: -155> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr: -154> 2024-03-04T18:04:12.069+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr: -153> 2024-03-04T18:04:12.069+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr: -152> 2024-03-04T18:04:12.069+0000 7f53f4dd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr: -151> 2024-03-04T18:04:12.069+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request con 0x556e05b6b190 auth_method 0 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr: -150> 2024-03-04T18:04:12.069+0000 7f53f3dcf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr: -149> 2024-03-04T18:04:12.069+0000 7f53f3dcf640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr: -148> 2024-03-04T18:04:12.069+0000 7f53f3dcf640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr: -147> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient(hunting): handle_auth_done global_id 6498 payload 824 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr: -146> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient: _finish_hunting 0 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr: -145> 2024-03-04T18:04:12.069+0000 7f53f45d0640 1 monclient: found mon.noname-a 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr: -144> 2024-03-04T18:04:12.069+0000 7f53f45d0640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr: -143> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr: -142> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr: -141> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: dump: 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.241 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.242 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.242 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.242 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.242 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.242 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.242 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.242 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.242 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.242 INFO:tasks.workunit.client.0.smithi137.stderr: -140> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: _finish_auth 0 2024-03-04T18:04:12.242 INFO:tasks.workunit.client.0.smithi137.stderr: -139> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: _check_auth_tickets 2024-03-04T18:04:12.242 INFO:tasks.workunit.client.0.smithi137.stderr: -138> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:12.242 INFO:tasks.workunit.client.0.smithi137.stderr: -137> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:12.242 INFO:tasks.workunit.client.0.smithi137.stderr: -136> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:12.242 INFO:tasks.workunit.client.0.smithi137.stderr: -135> 2024-03-04T18:04:12.069+0000 7f52c9ffb640 10 monclient: dump: 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr: -134> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 monclient: authenticate success, global_id 6498 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr: -133> 2024-03-04T18:04:12.069+0000 7f52cb7fe640 4 set_mon_vals no callback set 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr: -132> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: _renew_subs 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr: -131> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.243 INFO:tasks.workunit.client.0.smithi137.stderr: -130> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: _renew_subs 2024-03-04T18:04:12.244 INFO:tasks.workunit.client.0.smithi137.stderr: -129> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.244 INFO:tasks.workunit.client.0.smithi137.stderr: -128> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 1 librados: init done 2024-03-04T18:04:12.244 INFO:tasks.workunit.client.0.smithi137.stderr: -127> 2024-03-04T18:04:12.069+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command cr dump hook 0x556e05b6b6c8 2024-03-04T18:04:12.244 INFO:tasks.workunit.client.0.smithi137.stderr: -126> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.244 INFO:tasks.workunit.client.0.smithi137.stderr: -125> 2024-03-04T18:04:12.073+0000 7f52c9ffb640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:04:12.244 INFO:tasks.workunit.client.0.smithi137.stderr: -124> 2024-03-04T18:04:12.073+0000 7f52c9ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:12.244 INFO:tasks.workunit.client.0.smithi137.stderr: -123> 2024-03-04T18:04:12.073+0000 7f52c9ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:12.244 INFO:tasks.workunit.client.0.smithi137.stderr: -122> 2024-03-04T18:04:12.073+0000 7f53f3dcf640 10 monclient: get_auth_request con 0x7f52bc065940 auth_method 0 2024-03-04T18:04:12.244 INFO:tasks.workunit.client.0.smithi137.stderr: -121> 2024-03-04T18:04:12.073+0000 7f53f4dd1640 10 monclient: get_auth_request con 0x556e05b84ab0 auth_method 0 2024-03-04T18:04:12.244 INFO:tasks.workunit.client.0.smithi137.stderr: -120> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.244 INFO:tasks.workunit.client.0.smithi137.stderr: -119> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 realm 2024-03-04T18:04:12.244 INFO:tasks.workunit.client.0.smithi137.stderr: -118> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.244 INFO:tasks.workunit.client.0.smithi137.stderr: -117> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.244 INFO:tasks.workunit.client.0.smithi137.stderr: -116> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -115> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -114> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -113> 2024-03-04T18:04:12.073+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -112> 2024-03-04T18:04:12.073+0000 7f53f45d0640 10 monclient: get_auth_request con 0x556e05b88e40 auth_method 0 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -111> 2024-03-04T18:04:12.077+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -110> 2024-03-04T18:04:12.077+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -109> 2024-03-04T18:04:12.077+0000 7f53f3dcf640 10 monclient: get_auth_request con 0x556e05b8c560 auth_method 0 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -108> 2024-03-04T18:04:12.077+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -107> 2024-03-04T18:04:12.077+0000 7f53f6ab8980 20 searching for the correct realm 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -106> 2024-03-04T18:04:12.077+0000 7f53f4dd1640 10 monclient: get_auth_request con 0x556e05b912a0 auth_method 0 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -105> 2024-03-04T18:04:12.077+0000 7f53f45d0640 10 monclient: get_auth_request con 0x7f53ec020040 auth_method 0 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -104> 2024-03-04T18:04:12.081+0000 7f53f3dcf640 10 monclient: get_auth_request con 0x7f53e40fa060 auth_method 0 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -103> 2024-03-04T18:04:12.081+0000 7f53f3dcf640 0 -- 172.21.15.137:0/1600796126 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f53e40fa060 msgr2=0x7f53e40a5300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -102> 2024-03-04T18:04:12.081+0000 7f52c9ffb640 1 client.6498.objecter ms_handle_reset 0x7f53e40fa060 session 0x7f53e40c7070 osd.0 2024-03-04T18:04:12.245 INFO:tasks.workunit.client.0.smithi137.stderr: -101> 2024-03-04T18:04:12.081+0000 7f52c9ffb640 10 monclient: _renew_subs 2024-03-04T18:04:12.246 INFO:tasks.workunit.client.0.smithi137.stderr: -100> 2024-03-04T18:04:12.081+0000 7f52c9ffb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:12.246 INFO:tasks.workunit.client.0.smithi137.stderr: -99> 2024-03-04T18:04:12.081+0000 7f53f3dcf640 10 monclient: get_auth_request con 0x7f52bc075460 auth_method 0 2024-03-04T18:04:12.246 INFO:tasks.workunit.client.0.smithi137.stderr: -98> 2024-03-04T18:04:12.081+0000 7f53f4dd1640 10 monclient: get_auth_request con 0x7f53ec01f920 auth_method 0 2024-03-04T18:04:12.246 INFO:tasks.workunit.client.0.smithi137.stderr: -97> 2024-03-04T18:04:12.085+0000 7f53f45d0640 10 monclient: get_auth_request con 0x7f53e40f1020 auth_method 0 2024-03-04T18:04:12.246 INFO:tasks.workunit.client.0.smithi137.stderr: -96> 2024-03-04T18:04:12.085+0000 7f53f3dcf640 0 -- 172.21.15.137:0/1600796126 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x556e05b8c560 msgr2=0x556e05b8e970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:12.246 INFO:tasks.workunit.client.0.smithi137.stderr: -95> 2024-03-04T18:04:12.085+0000 7f52c9ffb640 1 client.6498.objecter ms_handle_reset 0x556e05b8c560 session 0x556e05b886e0 osd.3 2024-03-04T18:04:12.246 INFO:tasks.workunit.client.0.smithi137.stderr: -94> 2024-03-04T18:04:12.085+0000 7f53f3dcf640 10 monclient: get_auth_request con 0x7f53e40fa060 auth_method 0 2024-03-04T18:04:12.246 INFO:tasks.workunit.client.0.smithi137.stderr: -93> 2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:12.246 INFO:tasks.workunit.client.0.smithi137.stderr: -92> 2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:12.246 INFO:tasks.workunit.client.0.smithi137.stderr: -91> 2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:12.246 INFO:tasks.workunit.client.0.smithi137.stderr: -90> 2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:12.246 INFO:tasks.workunit.client.0.smithi137.stderr: -89> 2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:12.246 INFO:tasks.workunit.client.0.smithi137.stderr: -88> 2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:12.246 INFO:tasks.workunit.client.0.smithi137.stderr: -87> 2024-03-04T18:04:12.093+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -86> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -85> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -84> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -83> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -82> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -81> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 zone default found 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -80> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 4 Realm: () 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -79> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -78> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -77> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -76> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 zonegroup default 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -75> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -74> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -73> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.247 INFO:tasks.workunit.client.0.smithi137.stderr: -72> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.248 INFO:tasks.workunit.client.0.smithi137.stderr: -71> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.248 INFO:tasks.workunit.client.0.smithi137.stderr: -70> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.248 INFO:tasks.workunit.client.0.smithi137.stderr: -69> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 started sync module instance, tier type = 2024-03-04T18:04:12.248 INFO:tasks.workunit.client.0.smithi137.stderr: -68> 2024-03-04T18:04:12.097+0000 7f53f6ab8980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:12.248 INFO:tasks.workunit.client.0.smithi137.stderr: -67> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=0 2024-03-04T18:04:12.248 INFO:tasks.workunit.client.0.smithi137.stderr: -66> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=1 2024-03-04T18:04:12.248 INFO:tasks.workunit.client.0.smithi137.stderr: -65> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=2 2024-03-04T18:04:12.248 INFO:tasks.workunit.client.0.smithi137.stderr: -64> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=3 2024-03-04T18:04:12.248 INFO:tasks.workunit.client.0.smithi137.stderr: -63> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=4 2024-03-04T18:04:12.248 INFO:tasks.workunit.client.0.smithi137.stderr: -62> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=5 2024-03-04T18:04:12.248 INFO:tasks.workunit.client.0.smithi137.stderr: -61> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=6 2024-03-04T18:04:12.248 INFO:tasks.workunit.client.0.smithi137.stderr: -60> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 20 add_watcher() i=7 2024-03-04T18:04:12.248 INFO:tasks.workunit.client.0.smithi137.stderr: -59> 2024-03-04T18:04:12.117+0000 7f53f6ab8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:12.248 INFO:tasks.workunit.client.0.smithi137.stderr: -58> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding auth protocol: cephx 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -57> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding auth protocol: cephx 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -56> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding auth protocol: cephx 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -55> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: secure 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -54> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: crc 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -53> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: secure 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -52> 2024-03-04T18:04:12.121+0000 7f52ab7de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -51> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: crc 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -50> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: secure 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -49> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: crc 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -48> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: crc 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -47> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: secure 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -46> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: crc 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -45> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: secure 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -44> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: crc 2024-03-04T18:04:12.249 INFO:tasks.workunit.client.0.smithi137.stderr: -43> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 AuthRegistry(0x7fff0b3db390) adding con mode: secure 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -42> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -41> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -40> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -39> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command cache list hook 0x556e05b607c0 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -38> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command cache inspect hook 0x556e05b607c0 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -37> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command cache erase hook 0x556e05b607c0 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -36> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command cache zap hook 0x556e05b607c0 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -35> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 note: GC not initialized 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -34> 2024-03-04T18:04:12.121+0000 7f52aa7dc640 20 reqs_thread_entry: start 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -33> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command sync trace show hook 0x7f53e4004470 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -32> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command sync trace history hook 0x7f53e4004470 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -31> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command sync trace active hook 0x7f53e4004470 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -30> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 5 asok(0x556e058b3130) register_command sync trace active_short hook 0x7f53e4004470 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -29> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:12.250 INFO:tasks.workunit.client.0.smithi137.stderr: -28> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 Filter name: none 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -27> 2024-03-04T18:04:12.121+0000 7f52a37fe640 20 reqs_thread_entry: start 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -26> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -25> 2024-03-04T18:04:12.121+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -24> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -23> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -22> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -21> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 : miss 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -20> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -19> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -18> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -17> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -16> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -15> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -14> 2024-03-04T18:04:12.125+0000 7f53f6ab8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 2024-03-04T18:04:12.251 INFO:tasks.workunit.client.0.smithi137.stderr: -13> 2024-03-04T18:04:12.133+0000 7f53f6ab8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:12.252 INFO:tasks.workunit.client.0.smithi137.stderr: -12> 2024-03-04T18:04:12.133+0000 7f53f6ab8980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:12.252 INFO:tasks.workunit.client.0.smithi137.stderr: -11> 2024-03-04T18:04:12.133+0000 7f53f6ab8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:12.252 INFO:tasks.workunit.client.0.smithi137.stderr: -10> 2024-03-04T18:04:12.133+0000 7f53f6ab8980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24, ofs0, ns] 2024-03-04T18:04:12.252 INFO:tasks.workunit.client.0.smithi137.stderr: -9> 2024-03-04T18:04:12.133+0000 7f52cb7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627868 cookie 93931030799136 notifier 6498 bl.length()=949 2024-03-04T18:04:12.252 INFO:tasks.workunit.client.0.smithi137.stderr: -8> 2024-03-04T18:04:12.133+0000 7f52cb7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:12.252 INFO:tasks.workunit.client.0.smithi137.stderr: -7> 2024-03-04T18:04:12.133+0000 7f52cb7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:12.252 INFO:tasks.workunit.client.0.smithi137.stderr: -6> 2024-03-04T18:04:12.133+0000 7f52cb7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:12.252 INFO:tasks.workunit.client.0.smithi137.stderr: -5> 2024-03-04T18:04:12.137+0000 7f53f6ab8980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): orig_sources= new_sources= 2024-03-04T18:04:12.252 INFO:tasks.workunit.client.0.smithi137.stderr: -4> 2024-03-04T18:04:12.137+0000 7f53f6ab8980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): potential sources added=[] removed=[] 2024-03-04T18:04:12.252 INFO:tasks.workunit.client.0.smithi137.stderr: -3> 2024-03-04T18:04:12.137+0000 7f53f6ab8980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): orig_dests= new_dests= 2024-03-04T18:04:12.252 INFO:tasks.workunit.client.0.smithi137.stderr: -2> 2024-03-04T18:04:12.137+0000 7f53f6ab8980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): potential dests added=[] removed=[] 2024-03-04T18:04:12.252 INFO:tasks.workunit.client.0.smithi137.stderr: -1> 2024-03-04T18:04:12.137+0000 7f53f6ab8980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f53f6ab8980 time 2024-03-04T18:04:12.141563+0000 2024-03-04T18:04:12.252 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f53fa38dd7b] 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f53fa38df2f] 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x556e0364d281] 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x556e03659ced] 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: 5: main() 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f53f91f8d90] 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: 7: __libc_start_main() 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: 8: _start() 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: 0> 2024-03-04T18:04:12.137+0000 7f53f6ab8980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7f53f6ab8980 thread_name:radosgw-admin 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.253 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:12.254 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f53f9211520] 2024-03-04T18:04:12.254 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:04:12.254 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:04:12.254 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:04:12.254 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f53fa38ddd5] 2024-03-04T18:04:12.254 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f53fa38df2f] 2024-03-04T18:04:12.254 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x556e0364d281] 2024-03-04T18:04:12.254 INFO:tasks.workunit.client.0.smithi137.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x29d) [0x556e03659ced] 2024-03-04T18:04:12.254 INFO:tasks.workunit.client.0.smithi137.stderr: 9: main() 2024-03-04T18:04:12.254 INFO:tasks.workunit.client.0.smithi137.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f53f91f8d90] 2024-03-04T18:04:12.254 INFO:tasks.workunit.client.0.smithi137.stderr: 11: __libc_start_main() 2024-03-04T18:04:12.254 INFO:tasks.workunit.client.0.smithi137.stderr: 12: _start() 2024-03-04T18:04:12.254 INFO:tasks.workunit.client.0.smithi137.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:04:12.254 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr:--- logging levels --- 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 none 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 lockdep 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 context 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 crush 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_balancer 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_locker 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log_expire 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_migrator 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 buffer 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 timer 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 filer 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 striper 2024-03-04T18:04:12.255 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 objecter 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rados 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_mirror 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_replay 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_pwl 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 journaler 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objectcacher 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 client 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 osd 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 optracker 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objclass 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 filestore 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 journal 2024-03-04T18:04:12.256 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 ms 2024-03-04T18:04:12.257 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mon 2024-03-04T18:04:12.257 INFO:tasks.workunit.client.0.smithi137.stderr: 0/10 monc 2024-03-04T18:04:12.257 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 paxos 2024-03-04T18:04:12.257 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 tp 2024-03-04T18:04:12.257 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 auth 2024-03-04T18:04:12.257 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 crypto 2024-03-04T18:04:12.257 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 finisher 2024-03-04T18:04:12.257 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 reserver 2024-03-04T18:04:12.257 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 heartbeatmap 2024-03-04T18:04:12.257 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 perfcounter 2024-03-04T18:04:12.257 INFO:tasks.workunit.client.0.smithi137.stderr: 20/20 rgw 2024-03-04T18:04:12.257 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_sync 2024-03-04T18:04:12.257 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_datacache 2024-03-04T18:04:12.257 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_access 2024-03-04T18:04:12.258 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:04:12.258 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_flight 2024-03-04T18:04:12.258 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:04:12.258 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 javaclient 2024-03-04T18:04:12.258 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 asok 2024-03-04T18:04:12.258 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 throttle 2024-03-04T18:04:12.258 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 refs 2024-03-04T18:04:12.258 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 compressor 2024-03-04T18:04:12.258 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluestore 2024-03-04T18:04:12.258 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluefs 2024-03-04T18:04:12.258 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 bdev 2024-03-04T18:04:12.258 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 kstore 2024-03-04T18:04:12.258 INFO:tasks.workunit.client.0.smithi137.stderr: 4/ 5 rocksdb 2024-03-04T18:04:12.258 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 fuse 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 2/ 5 mgr 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mgrc 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 dpdk 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 eventtrace 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 prioritycache 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 test 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephsqlite 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_onode 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_odata 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_omap 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_tm 2024-03-04T18:04:12.259 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_t 2024-03-04T18:04:12.260 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:04:12.260 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_epm 2024-03-04T18:04:12.260 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_lba 2024-03-04T18:04:12.260 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:04:12.260 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cache 2024-03-04T18:04:12.260 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_journal 2024-03-04T18:04:12.260 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_device 2024-03-04T18:04:12.260 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_backref 2024-03-04T18:04:12.260 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 alienstore 2024-03-04T18:04:12.260 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mclock 2024-03-04T18:04:12.260 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cyanstore 2024-03-04T18:04:12.260 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 ceph_exporter 2024-03-04T18:04:12.260 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 memstore 2024-03-04T18:04:12.260 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 trace 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr: -2/-2 (syslog threshold) 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr: 99/99 (stderr threshold) 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52a37fe640 / http_manager 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52aa7dc640 / http_manager 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52ab7de640 / rgw_dt_lg_renew 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52c9ffb640 / ms_dispatch 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52cb7fe640 / io_context_pool 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52d9ffb640 / ms_dispatch 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52e0e09640 / ms_dispatch 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr: 7f52e260c640 / io_context_pool 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr: 7f53f3dcf640 / msgr-worker-2 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr: 7f53f45d0640 / msgr-worker-1 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr: 7f53f4dd1640 / msgr-worker-0 2024-03-04T18:04:12.261 INFO:tasks.workunit.client.0.smithi137.stderr: 7f53f55d2640 / io_context_pool 2024-03-04T18:04:12.262 INFO:tasks.workunit.client.0.smithi137.stderr: 7f53f6ab8980 / radosgw-admin 2024-03-04T18:04:12.262 INFO:tasks.workunit.client.0.smithi137.stderr: max_recent 500 2024-03-04T18:04:12.262 INFO:tasks.workunit.client.0.smithi137.stderr: max_new 1000 2024-03-04T18:04:12.262 INFO:tasks.workunit.client.0.smithi137.stderr: log_file /var/lib/ceph/crash/2024-03-04T18:04:12.143479Z_52f87b31-2e2b-475b-84e2-e222bcdcf9ec/log 2024-03-04T18:04:12.262 INFO:tasks.workunit.client.0.smithi137.stderr:--- end dump of recent events --- 2024-03-04T18:04:12.262 INFO:tasks.workunit.client.0.smithi137.stderr:Aborted 2024-03-04T18:04:12.262 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-03-04T18:04:12.263 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:12.263 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:12.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.217+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.221+0000 7f0f8298b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.221+0000 7f0f8298b980 20 realm 2024-03-04T18:04:12.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.221+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.221+0000 7f0f8298b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.221+0000 7f0f8298b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:12.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.221+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.221+0000 7f0f8298b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.221+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.221+0000 7f0f8298b980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:12.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.221+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.221+0000 7f0f8298b980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:12.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.221+0000 7f0f8298b980 20 searching for the correct realm 2024-03-04T18:04:12.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.229+0000 7f0f80ca4640 0 -- 172.21.15.137:0/2295727769 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f0f780410b0 msgr2=0x7f0f78018830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:12.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.445+0000 7f0f80ca4640 0 -- 172.21.15.137:0/2295727769 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55cf2ab8be40 msgr2=0x55cf2ab8e300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:12.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.653+0000 7f0f8298b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:12.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.653+0000 7f0f8298b980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:12.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.653+0000 7f0f8298b980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:12.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.653+0000 7f0f8298b980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:12.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.653+0000 7f0f8298b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:12.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.653+0000 7f0f8298b980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:12.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.657 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:12.658 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.658 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:12.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 20 zone default found 2024-03-04T18:04:12.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 4 Realm: () 2024-03-04T18:04:12.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:12.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:12.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:12.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 20 zonegroup default 2024-03-04T18:04:12.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.659 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.660 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.657+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.660 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.661+0000 7f0f8298b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.660 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.661+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.662 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.661+0000 7f0f8298b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.662 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.661+0000 7f0f8298b980 20 started sync module instance, tier type = 2024-03-04T18:04:12.662 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.661+0000 7f0f8298b980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:12.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.685+0000 7f0f8298b980 20 add_watcher() i=0 2024-03-04T18:04:12.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.685+0000 7f0f8298b980 20 add_watcher() i=1 2024-03-04T18:04:12.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.685+0000 7f0f8298b980 20 add_watcher() i=2 2024-03-04T18:04:12.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.685+0000 7f0f8298b980 20 add_watcher() i=3 2024-03-04T18:04:12.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.685+0000 7f0f8298b980 20 add_watcher() i=4 2024-03-04T18:04:12.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.685+0000 7f0f8298b980 20 add_watcher() i=5 2024-03-04T18:04:12.685 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.685+0000 7f0f8298b980 20 add_watcher() i=6 2024-03-04T18:04:12.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.685+0000 7f0f8298b980 20 add_watcher() i=7 2024-03-04T18:04:12.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.685+0000 7f0f8298b980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:12.692 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.693+0000 7f0e36fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:12.693 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.693+0000 7f0f8298b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:12.693 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.693+0000 7f0f8298b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:12.693 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.693+0000 7f0f8298b980 5 note: GC not initialized 2024-03-04T18:04:12.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.693+0000 7f0e35fdb640 20 reqs_thread_entry: start 2024-03-04T18:04:12.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.693+0000 7f0f8298b980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:12.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.693+0000 7f0f8298b980 20 Filter name: none 2024-03-04T18:04:12.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.693+0000 7f0da77fe640 20 reqs_thread_entry: start 2024-03-04T18:04:12.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.693+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.693+0000 7f0f8298b980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:12.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.693+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.697+0000 7f0f8298b980 20 rados_obj.operate() r=0 bl.length=417 2024-03-04T18:04:12.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.697+0000 7f0f8298b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:12.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.697+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.697+0000 7f0f8298b980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:12.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.697+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.697+0000 7f0f8298b980 20 rados_obj.operate() r=0 bl.length=417 2024-03-04T18:04:12.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.697+0000 7f0f8298b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:12.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.697+0000 7f0f8298b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.701+0000 7f0f8298b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.701 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.701+0000 7f0e36fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:12.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.705+0000 7f0f8298b980 20 remove_watcher() i=0 2024-03-04T18:04:12.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.705+0000 7f0f8298b980 2 removed watcher, disabling cache 2024-03-04T18:04:12.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.705+0000 7f0f8298b980 20 remove_watcher() i=1 2024-03-04T18:04:12.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.709+0000 7f0f8298b980 20 remove_watcher() i=2 2024-03-04T18:04:12.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.709+0000 7f0f8298b980 20 remove_watcher() i=3 2024-03-04T18:04:12.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.713+0000 7f0f8298b980 20 remove_watcher() i=4 2024-03-04T18:04:12.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.713+0000 7f0f8298b980 20 remove_watcher() i=5 2024-03-04T18:04:12.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.717+0000 7f0f8298b980 20 remove_watcher() i=6 2024-03-04T18:04:12.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.721+0000 7f0f8298b980 20 remove_watcher() i=7 2024-03-04T18:04:12.735 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.735 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:04:12.736 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: abort-at-block-writes id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:04:12.736 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:12.737 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:12.737 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:12.772 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:12.772 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:12.850 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:04:12.885 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:12.885 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:12.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.929+0000 7f7734170980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.929+0000 7f7734170980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.929+0000 7f7734170980 20 realm 2024-03-04T18:04:12.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.929+0000 7f7734170980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.933+0000 7f7734170980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.933+0000 7f7734170980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:12.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.933+0000 7f7734170980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.932 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.933+0000 7f7734170980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.933 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.933+0000 7f7734170980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.933 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.933+0000 7f7734170980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:12.933 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.933+0000 7f7734170980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.934 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.933+0000 7f7734170980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:12.934 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.933+0000 7f7734170980 20 searching for the correct realm 2024-03-04T18:04:12.938 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.937+0000 7f7731487640 0 -- 172.21.15.137:0/896667970 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f772c0ee070 msgr2=0x7f772c0f07e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:12.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.945+0000 7f7731487640 0 -- 172.21.15.137:0/896667970 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x557f04271f70 msgr2=0x557f04274380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:12.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:12.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:12.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:12.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:12.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:12.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:12.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:12.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:12.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 zone default found 2024-03-04T18:04:12.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 4 Realm: () 2024-03-04T18:04:12.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:12.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:12.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:12.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 zonegroup default 2024-03-04T18:04:12.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:12.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 started sync module instance, tier type = 2024-03-04T18:04:12.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.953+0000 7f7734170980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:12.973 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.973+0000 7f7734170980 20 add_watcher() i=0 2024-03-04T18:04:12.973 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.973+0000 7f7734170980 20 add_watcher() i=1 2024-03-04T18:04:12.973 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.973+0000 7f7734170980 20 add_watcher() i=2 2024-03-04T18:04:12.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.973+0000 7f7734170980 20 add_watcher() i=3 2024-03-04T18:04:12.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.973+0000 7f7734170980 20 add_watcher() i=4 2024-03-04T18:04:12.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.973+0000 7f7734170980 20 add_watcher() i=5 2024-03-04T18:04:12.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.973+0000 7f7734170980 20 add_watcher() i=6 2024-03-04T18:04:12.975 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.973+0000 7f7734170980 20 add_watcher() i=7 2024-03-04T18:04:12.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.973+0000 7f7734170980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:12.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.977+0000 7f75e37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:12.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.977+0000 7f7734170980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:12.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.977+0000 7f7734170980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:12.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.977+0000 7f7734170980 5 note: GC not initialized 2024-03-04T18:04:12.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.977+0000 7f75e27fc640 20 reqs_thread_entry: start 2024-03-04T18:04:12.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.977+0000 7f7734170980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:12.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.977+0000 7f7734170980 20 Filter name: none 2024-03-04T18:04:12.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.977+0000 7f7553fff640 20 reqs_thread_entry: start 2024-03-04T18:04:12.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.977+0000 7f7734170980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-03-04T18:04:12.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.977+0000 7f7734170980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.981+0000 7f7734170980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:12.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.981+0000 7f7734170980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-03-04T18:04:12.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.981+0000 7f7734170980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-03-04T18:04:12.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.981+0000 7f7734170980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 : miss 2024-03-04T18:04:12.981 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.981+0000 7f7734170980 20 rados->read ofs=0 len=0 2024-03-04T18:04:12.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.981+0000 7f7734170980 20 rados_obj.operate() r=0 bl.length=417 2024-03-04T18:04:12.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.981+0000 7f7734170980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:12.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.981+0000 7f7734170980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:12.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.981+0000 7f7734170980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:12.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.981+0000 7f7734170980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:12.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.981+0000 7f7734170980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 2024-03-04T18:04:12.983 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.981+0000 7f7734170980 0 INFO: RGWReshardLock::lock found lock on abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to be held by another RGW process; skipping for now 2024-03-04T18:04:12.985 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.985+0000 7f75e37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:12.988 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.989+0000 7f7734170980 20 remove_watcher() i=0 2024-03-04T18:04:12.989 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.989+0000 7f7734170980 2 removed watcher, disabling cache 2024-03-04T18:04:12.991 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.989+0000 7f7734170980 20 remove_watcher() i=1 2024-03-04T18:04:12.992 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.993+0000 7f7734170980 20 remove_watcher() i=2 2024-03-04T18:04:12.994 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.993+0000 7f7734170980 20 remove_watcher() i=3 2024-03-04T18:04:12.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.997+0000 7f7734170980 20 remove_watcher() i=4 2024-03-04T18:04:12.998 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:12.997+0000 7f7734170980 20 remove_watcher() i=5 2024-03-04T18:04:13.001 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:13.001+0000 7f7734170980 20 remove_watcher() i=6 2024-03-04T18:04:13.003 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:13.001+0000 7f7734170980 20 remove_watcher() i=7 2024-03-04T18:04:13.017 INFO:tasks.workunit.client.0.smithi137.stderr:waiting 30 seconds for reshard lock to expire... 2024-03-04T18:04:43.039 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:04:43.074 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:43.074 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:43.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.112+0000 7fa6965a7980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.116+0000 7fa6965a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.116+0000 7fa6965a7980 20 realm 2024-03-04T18:04:43.118 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.116+0000 7fa6965a7980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.116+0000 7fa6965a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.116+0000 7fa6965a7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:43.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.116+0000 7fa6965a7980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.116+0000 7fa6965a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.119 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.116+0000 7fa6965a7980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.116+0000 7fa6965a7980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:43.120 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.116+0000 7fa6965a7980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.120+0000 7fa6965a7980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:43.121 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.120+0000 7fa6965a7980 20 searching for the correct realm 2024-03-04T18:04:43.126 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.124+0000 7fa6948c0640 0 -- 172.21.15.137:0/36220752 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fa6840ac800 msgr2=0x7fa6840f1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:43.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.340+0000 7fa6948c0640 0 -- 172.21.15.137:0/36220752 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55cad91c24d0 msgr2=0x55cad91c4990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.348+0000 7fa6965a7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.348+0000 7fa6965a7980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.348+0000 7fa6965a7980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.348+0000 7fa6965a7980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.348+0000 7fa6965a7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.348+0000 7fa6965a7980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:43.350 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.348+0000 7fa6965a7980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.351 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.348+0000 7fa6965a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.351 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.348+0000 7fa6965a7980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.351 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.348+0000 7fa6965a7980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:43.351 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.348+0000 7fa6965a7980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:43.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 20 zone default found 2024-03-04T18:04:43.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 4 Realm: () 2024-03-04T18:04:43.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:43.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:43.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:43.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 20 zonegroup default 2024-03-04T18:04:43.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.353 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 20 started sync module instance, tier type = 2024-03-04T18:04:43.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.352+0000 7fa6965a7980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:43.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.376+0000 7fa6965a7980 20 add_watcher() i=0 2024-03-04T18:04:43.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.376+0000 7fa6965a7980 20 add_watcher() i=1 2024-03-04T18:04:43.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.376+0000 7fa6965a7980 20 add_watcher() i=2 2024-03-04T18:04:43.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.376+0000 7fa6965a7980 20 add_watcher() i=3 2024-03-04T18:04:43.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.376+0000 7fa6965a7980 20 add_watcher() i=4 2024-03-04T18:04:43.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.376+0000 7fa6965a7980 20 add_watcher() i=5 2024-03-04T18:04:43.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.376+0000 7fa6965a7980 20 add_watcher() i=6 2024-03-04T18:04:43.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.380+0000 7fa6965a7980 20 add_watcher() i=7 2024-03-04T18:04:43.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.380+0000 7fa6965a7980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:43.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.384+0000 7fa54afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:43.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.384+0000 7fa6965a7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:43.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.384+0000 7fa6965a7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:43.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.384+0000 7fa6965a7980 5 note: GC not initialized 2024-03-04T18:04:43.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.384+0000 7fa549fdb640 20 reqs_thread_entry: start 2024-03-04T18:04:43.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.384+0000 7fa6965a7980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:43.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.384+0000 7fa6965a7980 20 Filter name: none 2024-03-04T18:04:43.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.384+0000 7fa4bb7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:43.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.384+0000 7fa6965a7980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-03-04T18:04:43.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.384+0000 7fa6965a7980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.388+0000 7fa6965a7980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:43.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.388+0000 7fa6965a7980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-03-04T18:04:43.388 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.388+0000 7fa6965a7980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-03-04T18:04:43.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.388+0000 7fa6965a7980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 : miss 2024-03-04T18:04:43.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.388+0000 7fa6965a7980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.388+0000 7fa6965a7980 20 rados_obj.operate() r=0 bl.length=417 2024-03-04T18:04:43.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.388+0000 7fa6965a7980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:43.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.388+0000 7fa6965a7980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:43.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.388+0000 7fa6965a7980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:43.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.388+0000 7fa6965a7980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:43.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.388+0000 7fa6965a7980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 2024-03-04T18:04:43.392 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.388+0000 7fa6965a7980 10 init_target_layout removing existing target index objects from a previous reshard attempt 2024-03-04T18:04:43.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.400+0000 7fa6965a7980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:43.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.400+0000 7fa6965a7980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:43.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.400+0000 7fa6965a7980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:43.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.400+0000 7fa6965a7980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24, ofs0, ns] 2024-03-04T18:04:43.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.400+0000 7fa56b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627869 cookie 94329714271344 notifier 6555 bl.length()=949 2024-03-04T18:04:43.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.400+0000 7fa56b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:43.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.400+0000 7fa56b7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:43.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.400+0000 7fa56b7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:43.405 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.404+0000 7fa6965a7980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): orig_sources= new_sources= 2024-03-04T18:04:43.405 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.404+0000 7fa6965a7980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): potential sources added=[] removed=[] 2024-03-04T18:04:43.405 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.404+0000 7fa6965a7980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): orig_dests= new_dests= 2024-03-04T18:04:43.405 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.404+0000 7fa6965a7980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): potential dests added=[] removed=[] 2024-03-04T18:04:43.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.408+0000 7fa6965a7980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24.2.0 2024-03-04T18:04:43.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.408+0000 7fa6965a7980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24.2.0 2024-03-04T18:04:43.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.408+0000 7fa6965a7980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24.2.1 2024-03-04T18:04:43.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.408+0000 7fa6965a7980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24.2.1 2024-03-04T18:04:43.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.408+0000 7fa6965a7980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24.0 2024-03-04T18:04:43.416 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.412+0000 7fa6965a7980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 : hit (requested=0x17, cached=0x17) 2024-03-04T18:04:43.416 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.412+0000 7fa6965a7980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 2024-03-04T18:04:43.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.416+0000 7fa6965a7980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:43.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.416+0000 7fa6965a7980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:43.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.416+0000 7fa6965a7980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:43.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.416+0000 7fa6965a7980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24, ofs0, ns] 2024-03-04T18:04:43.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.416+0000 7fa56b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627870 cookie 94329714271344 notifier 6555 bl.length()=917 2024-03-04T18:04:43.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.416+0000 7fa56b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 info.flags=0x17 2024-03-04T18:04:43.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.416+0000 7fa56b7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 to cache LRU end 2024-03-04T18:04:43.419 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.416+0000 7fa56b7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:43.421 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.420+0000 7fa6965a7980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): orig_sources= new_sources= 2024-03-04T18:04:43.422 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.420+0000 7fa6965a7980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): potential sources added=[] removed=[] 2024-03-04T18:04:43.422 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.420+0000 7fa6965a7980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): orig_dests= new_dests= 2024-03-04T18:04:43.422 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.420+0000 7fa6965a7980 20 handle_bi_update(): bucket=:abort-at-block-writes[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24]): potential dests added=[] removed=[] 2024-03-04T18:04:43.425 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.424+0000 7fa6965a7980 1 execute INFO: reshard of bucket "abort-at-block-writes" completed successfully 2024-03-04T18:04:43.429 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.428+0000 7fa54afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:43.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.432+0000 7fa6965a7980 20 remove_watcher() i=0 2024-03-04T18:04:43.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.432+0000 7fa6965a7980 2 removed watcher, disabling cache 2024-03-04T18:04:43.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.432+0000 7fa6965a7980 20 remove_watcher() i=1 2024-03-04T18:04:43.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.436+0000 7fa6965a7980 20 remove_watcher() i=2 2024-03-04T18:04:43.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.436+0000 7fa6965a7980 20 remove_watcher() i=3 2024-03-04T18:04:43.442 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.440+0000 7fa6965a7980 20 remove_watcher() i=4 2024-03-04T18:04:43.444 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.444+0000 7fa6965a7980 20 remove_watcher() i=5 2024-03-04T18:04:43.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.444+0000 7fa6965a7980 20 remove_watcher() i=6 2024-03-04T18:04:43.449 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.448+0000 7fa6965a7980 20 remove_watcher() i=7 2024-03-04T18:04:43.470 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-03-04T18:04:43.501 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:43.501 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:43.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.544+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.548+0000 7ff4cb45b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.548+0000 7ff4cb45b980 20 realm 2024-03-04T18:04:43.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.548+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.548+0000 7ff4cb45b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.548+0000 7ff4cb45b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:43.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.548+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.548+0000 7ff4cb45b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.548+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.548+0000 7ff4cb45b980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:43.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.548+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.552+0000 7ff4cb45b980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:43.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.552+0000 7ff4cb45b980 20 searching for the correct realm 2024-03-04T18:04:43.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.556+0000 7ff4c9774640 0 -- 172.21.15.137:0/852090674 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7ff4c0074040 msgr2=0x7ff4c00880c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:43.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.772+0000 7ff4c9774640 0 -- 172.21.15.137:0/852090674 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x56424af2e060 msgr2=0x56424af30520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:43.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.780+0000 7ff4cb45b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:43.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.780+0000 7ff4cb45b980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:43.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.780+0000 7ff4cb45b980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:43.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.780+0000 7ff4cb45b980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:43.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.780+0000 7ff4cb45b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:43.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.780+0000 7ff4cb45b980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:43.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.780+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.780+0000 7ff4cb45b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.780+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:43.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.785 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:43.785 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 20 zone default found 2024-03-04T18:04:43.785 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 4 Realm: () 2024-03-04T18:04:43.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:43.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:43.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:43.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 20 zonegroup default 2024-03-04T18:04:43.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.787 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.787 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.787 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.788 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 20 started sync module instance, tier type = 2024-03-04T18:04:43.788 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.784+0000 7ff4cb45b980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:43.810 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.808+0000 7ff4cb45b980 20 add_watcher() i=0 2024-03-04T18:04:43.810 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.808+0000 7ff4cb45b980 20 add_watcher() i=1 2024-03-04T18:04:43.810 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.808+0000 7ff4cb45b980 20 add_watcher() i=2 2024-03-04T18:04:43.810 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.808+0000 7ff4cb45b980 20 add_watcher() i=3 2024-03-04T18:04:43.810 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.808+0000 7ff4cb45b980 20 add_watcher() i=4 2024-03-04T18:04:43.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.808+0000 7ff4cb45b980 20 add_watcher() i=5 2024-03-04T18:04:43.811 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.808+0000 7ff4cb45b980 20 add_watcher() i=6 2024-03-04T18:04:43.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.812+0000 7ff4cb45b980 20 add_watcher() i=7 2024-03-04T18:04:43.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.812+0000 7ff4cb45b980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:43.817 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.816+0000 7ff37efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:43.818 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.816+0000 7ff4cb45b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:43.818 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.816+0000 7ff4cb45b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:43.818 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.816+0000 7ff4cb45b980 5 note: GC not initialized 2024-03-04T18:04:43.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.816+0000 7ff37dfe3640 20 reqs_thread_entry: start 2024-03-04T18:04:43.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.816+0000 7ff4cb45b980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:43.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.816+0000 7ff4cb45b980 20 Filter name: none 2024-03-04T18:04:43.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.816+0000 7ff2ef7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:43.819 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.816+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.820 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.820+0000 7ff4cb45b980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:43.820 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.820+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.821 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.820+0000 7ff4cb45b980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:43.821 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.820+0000 7ff4cb45b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:43.821 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.820+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.822 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.820+0000 7ff4cb45b980 20 rados_obj.operate() r=0 bl.length=213 2024-03-04T18:04:43.822 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.820+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.823 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.820+0000 7ff4cb45b980 20 rados_obj.operate() r=0 bl.length=385 2024-03-04T18:04:43.823 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.820+0000 7ff4cb45b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:43.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.824+0000 7ff4cb45b980 20 rados->read ofs=0 len=0 2024-03-04T18:04:43.825 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.824+0000 7ff4cb45b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:43.826 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.824+0000 7ff37efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:43.831 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.828+0000 7ff4cb45b980 20 remove_watcher() i=0 2024-03-04T18:04:43.831 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.828+0000 7ff4cb45b980 2 removed watcher, disabling cache 2024-03-04T18:04:43.833 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.832+0000 7ff4cb45b980 20 remove_watcher() i=1 2024-03-04T18:04:43.835 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.832+0000 7ff4cb45b980 20 remove_watcher() i=2 2024-03-04T18:04:43.837 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.836+0000 7ff4cb45b980 20 remove_watcher() i=3 2024-03-04T18:04:43.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.836+0000 7ff4cb45b980 20 remove_watcher() i=4 2024-03-04T18:04:43.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.840+0000 7ff4cb45b980 20 remove_watcher() i=5 2024-03-04T18:04:43.844 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.844+0000 7ff4cb45b980 20 remove_watcher() i=6 2024-03-04T18:04:43.847 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:43.844+0000 7ff4cb45b980 20 remove_watcher() i=7 2024-03-04T18:04:43.861 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:43.861 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:04:43.862 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: abort-at-block-writes id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.24 num_objects: 19 size_kb: 1 num_shards: 2 2024-03-04T18:04:43.862 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:43.863 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:43.863 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:43.890 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:43.890 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:44.309 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:44.309 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:44.483 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: reshard bucket with EIO injected at commit_target_layout 2024-03-04T18:04:44.483 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:44.487 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:44.487 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:44.595 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:44.595 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:44.676 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:44.676 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:44.759 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:44.759 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:44.840 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:44.840 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:44.925 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:44.925 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:45.009 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.010 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:45.093 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.093 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:45.178 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.179 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:45.260 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.260 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:45.345 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.345 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:45.428 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.428 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:45.514 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.514 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:45.595 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.595 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:45.674 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.674 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:45.765 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.765 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:45.843 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.843 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:45.925 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:45.925 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:46.007 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:46.007 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:46.089 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:46.089 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:46.173 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:46.173 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:46.259 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:46.259 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:46.361 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-03-04T18:04:46.393 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:46.394 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:46.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.436+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.441 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.440+0000 7f71652ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.441 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.440+0000 7f71652ac980 20 realm 2024-03-04T18:04:46.441 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.440+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.441 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.440+0000 7f71652ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.442 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.440+0000 7f71652ac980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:46.442 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.440+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.442 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.440+0000 7f71652ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.442 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.440+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.442 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.440+0000 7f71652ac980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:46.442 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.440+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.646 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.644+0000 7f71652ac980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:46.646 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.644+0000 7f71652ac980 20 searching for the correct realm 2024-03-04T18:04:46.851 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.848+0000 7f7162dc4640 0 -- 172.21.15.137:0/3385798800 >> [v2:172.21.15.137:6808/3491386063,v1:172.21.15.137:6810/3491386063] conn(0x7f715c018040 msgr2=0x7f715c027980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:46.860 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.860+0000 7f7162dc4640 0 -- 172.21.15.137:0/3385798800 >> [v2:172.21.15.137:6824/1039931256,v1:172.21.15.137:6825/1039931256] conn(0x55679ea924a0 msgr2=0x55679ea94890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:46.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.876+0000 7f71652ac980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:46.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.876+0000 7f71652ac980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:46.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.876+0000 7f71652ac980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:46.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.876+0000 7f71652ac980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:46.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.876+0000 7f71652ac980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:46.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.876+0000 7f71652ac980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:46.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.876+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.876+0000 7f71652ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.876+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.876+0000 7f71652ac980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:46.879 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.876+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.880 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:46.880 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 20 zone default found 2024-03-04T18:04:46.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 4 Realm: () 2024-03-04T18:04:46.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:46.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:46.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:46.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 20 zonegroup default 2024-03-04T18:04:46.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.882 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 20 started sync module instance, tier type = 2024-03-04T18:04:46.883 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.880+0000 7f71652ac980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:46.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.904+0000 7f71652ac980 20 add_watcher() i=0 2024-03-04T18:04:46.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.904+0000 7f71652ac980 20 add_watcher() i=1 2024-03-04T18:04:46.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.904+0000 7f71652ac980 20 add_watcher() i=2 2024-03-04T18:04:46.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.904+0000 7f71652ac980 20 add_watcher() i=3 2024-03-04T18:04:46.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.904+0000 7f71652ac980 20 add_watcher() i=4 2024-03-04T18:04:46.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.904+0000 7f71652ac980 20 add_watcher() i=5 2024-03-04T18:04:46.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.904+0000 7f71652ac980 20 add_watcher() i=6 2024-03-04T18:04:46.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.904+0000 7f71652ac980 20 add_watcher() i=7 2024-03-04T18:04:46.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.904+0000 7f71652ac980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:46.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.908+0000 7f70197e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:46.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.912+0000 7f71652ac980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:46.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.912+0000 7f71652ac980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:46.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.912+0000 7f71652ac980 5 note: GC not initialized 2024-03-04T18:04:46.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.912+0000 7f6f93fff640 20 reqs_thread_entry: start 2024-03-04T18:04:46.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.912+0000 7f71652ac980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:46.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.912+0000 7f71652ac980 20 Filter name: none 2024-03-04T18:04:46.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.912+0000 7f6f91ffb640 20 reqs_thread_entry: start 2024-03-04T18:04:46.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.912+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.912+0000 7f71652ac980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:46.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.912+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.912+0000 7f71652ac980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:46.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.912+0000 7f71652ac980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:46.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.912+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.916 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.916+0000 7f71652ac980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:46.916 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.916+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.916 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.916+0000 7f71652ac980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:46.917 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.916+0000 7f71652ac980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:46.917 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.916+0000 7f71652ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:46.918 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.916+0000 7f71652ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:46.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.916+0000 7f70197e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:46.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.920+0000 7f71652ac980 20 remove_watcher() i=0 2024-03-04T18:04:46.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.920+0000 7f71652ac980 2 removed watcher, disabling cache 2024-03-04T18:04:46.926 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.924+0000 7f71652ac980 20 remove_watcher() i=1 2024-03-04T18:04:46.927 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.924+0000 7f71652ac980 20 remove_watcher() i=2 2024-03-04T18:04:46.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.928+0000 7f71652ac980 20 remove_watcher() i=3 2024-03-04T18:04:46.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.928+0000 7f71652ac980 20 remove_watcher() i=4 2024-03-04T18:04:46.934 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.932+0000 7f71652ac980 20 remove_watcher() i=5 2024-03-04T18:04:46.936 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.936+0000 7f71652ac980 20 remove_watcher() i=6 2024-03-04T18:04:46.938 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:46.936+0000 7f71652ac980 20 remove_watcher() i=7 2024-03-04T18:04:46.954 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:46.954 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:04:46.954 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-commit-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:04:46.955 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:46.955 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout 2024-03-04T18:04:46.986 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:46.986 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:47.027 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.024+0000 7f8ef6757980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.028+0000 7f8ef6757980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.028+0000 7f8ef6757980 20 realm 2024-03-04T18:04:47.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.028+0000 7f8ef6757980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.028+0000 7f8ef6757980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.028+0000 7f8ef6757980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:47.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.028+0000 7f8ef6757980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.028+0000 7f8ef6757980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.030 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.028+0000 7f8ef6757980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.031 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.028+0000 7f8ef6757980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.031 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.028+0000 7f8ef6757980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.031 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.028+0000 7f8ef6757980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:47.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.028+0000 7f8ef6757980 20 searching for the correct realm 2024-03-04T18:04:47.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.036+0000 7f8ef4a70640 0 -- 172.21.15.137:0/1449201325 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f8eec0fbdd0 msgr2=0x7f8eec10b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:47.253 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.252+0000 7f8ef4a70640 0 -- 172.21.15.137:0/1449201325 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x557d25a6c4c0 msgr2=0x557d25a6e980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.260+0000 7f8ef6757980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.260+0000 7f8ef6757980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.260+0000 7f8ef6757980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:47.262 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.260+0000 7f8ef6757980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.260+0000 7f8ef6757980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.260+0000 7f8ef6757980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.260+0000 7f8ef6757980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.260+0000 7f8ef6757980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.260+0000 7f8ef6757980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.260+0000 7f8ef6757980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.260+0000 7f8ef6757980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 20 zone default found 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 4 Realm: () 2024-03-04T18:04:47.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 20 zonegroup default 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 20 started sync module instance, tier type = 2024-03-04T18:04:47.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.264+0000 7f8ef6757980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.288+0000 7f8ef6757980 20 add_watcher() i=0 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.288+0000 7f8ef6757980 20 add_watcher() i=1 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.288+0000 7f8ef6757980 20 add_watcher() i=2 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.288+0000 7f8ef6757980 20 add_watcher() i=3 2024-03-04T18:04:47.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.288+0000 7f8ef6757980 20 add_watcher() i=4 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.288+0000 7f8ef6757980 20 add_watcher() i=5 2024-03-04T18:04:47.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.288+0000 7f8ef6757980 20 add_watcher() i=6 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.288+0000 7f8ef6757980 20 add_watcher() i=7 2024-03-04T18:04:47.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.288+0000 7f8ef6757980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:47.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.292+0000 7f8daafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.292+0000 7f8ef6757980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.292+0000 7f8ef6757980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.292+0000 7f8ef6757980 5 note: GC not initialized 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8da9fdb640 20 reqs_thread_entry: start 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:47.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 20 Filter name: none 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8d1b7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-03-04T18:04:47.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 : miss 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 info.flags=0x17 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 to cache LRU end 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:47.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.296+0000 7f8ef6757980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.304+0000 7f8ef6757980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 info.flags=0x17 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.304+0000 7f8ef6757980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 to cache LRU end 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.304+0000 7f8ef6757980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:47.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.304+0000 7f8ef6757980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25, ofs0, ns] 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.304+0000 7f8dcb7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627843 cookie 93995990966320 notifier 6263 bl.length()=965 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.304+0000 7f8dcb7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 info.flags=0x17 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.304+0000 7f8dcb7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 to cache LRU end 2024-03-04T18:04:47.306 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.304+0000 7f8dcb7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.308+0000 7f8ef6757980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25]): orig_sources= new_sources= 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.308+0000 7f8ef6757980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25]): potential sources added=[] removed=[] 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.308+0000 7f8ef6757980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25]): orig_dests= new_dests= 2024-03-04T18:04:47.309 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.308+0000 7f8ef6757980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25]): potential dests added=[] removed=[] 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.308+0000 7f8ef6757980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25.1.0 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.308+0000 7f8ef6757980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25.1.0 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.308+0000 7f8ef6757980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25.1.1 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.308+0000 7f8ef6757980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25.1.1 2024-03-04T18:04:47.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.308+0000 7f8ef6757980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25.0 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.316+0000 7f8ef6757980 -1 Injecting error=-5 at location=commit_target_layout 2024-03-04T18:04:47.317 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.316+0000 7f8ef6757980 0 ERROR: commit_reshard failed to commit target index layout: (5) Input/output error 2024-03-04T18:04:47.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.320+0000 7f8daafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.324+0000 7f8ef6757980 20 remove_watcher() i=0 2024-03-04T18:04:47.327 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.324+0000 7f8ef6757980 2 removed watcher, disabling cache 2024-03-04T18:04:47.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.328+0000 7f8ef6757980 20 remove_watcher() i=1 2024-03-04T18:04:47.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.328+0000 7f8ef6757980 20 remove_watcher() i=2 2024-03-04T18:04:47.334 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.332+0000 7f8ef6757980 20 remove_watcher() i=3 2024-03-04T18:04:47.335 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.332+0000 7f8ef6757980 20 remove_watcher() i=4 2024-03-04T18:04:47.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.336+0000 7f8ef6757980 20 remove_watcher() i=5 2024-03-04T18:04:47.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.340+0000 7f8ef6757980 20 remove_watcher() i=6 2024-03-04T18:04:47.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.340+0000 7f8ef6757980 20 remove_watcher() i=7 2024-03-04T18:04:47.359 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-03-04T18:04:47.391 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:47.391 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:47.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.432+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.437 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.436+0000 7f22baf80980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.437 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.436+0000 7f22baf80980 20 realm 2024-03-04T18:04:47.437 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.436+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.437 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.436+0000 7f22baf80980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.437 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.436+0000 7f22baf80980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:47.437 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.436+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.436+0000 7f22baf80980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.436+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.436+0000 7f22baf80980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.436+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.436+0000 7f22baf80980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:47.439 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.436+0000 7f22baf80980 20 searching for the correct realm 2024-03-04T18:04:47.443 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.440+0000 7f22b8a98640 0 -- 172.21.15.137:0/1296454339 >> [v2:172.21.15.137:6824/1039931256,v1:172.21.15.137:6825/1039931256] conn(0x56391b8b3d00 msgr2=0x56391b8b61c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:47.448 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.448+0000 7f22b8a98640 0 -- 172.21.15.137:0/1296454339 >> [v2:172.21.15.173:6805/879938785,v1:172.21.15.173:6808/879938785] conn(0x7f22ac03eb20 msgr2=0x7f22ac04c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:47.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:47.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:47.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:47.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:47.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:47.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:47.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:47.672 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 zone default found 2024-03-04T18:04:47.672 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 4 Realm: () 2024-03-04T18:04:47.672 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:47.672 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:47.672 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:47.672 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 zonegroup default 2024-03-04T18:04:47.672 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.668+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.672+0000 7f22baf80980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.672+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.672+0000 7f22baf80980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.672+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.672+0000 7f22baf80980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.672+0000 7f22baf80980 20 started sync module instance, tier type = 2024-03-04T18:04:47.674 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.672+0000 7f22baf80980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:47.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.696+0000 7f22baf80980 20 add_watcher() i=0 2024-03-04T18:04:47.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.696+0000 7f22baf80980 20 add_watcher() i=1 2024-03-04T18:04:47.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.696+0000 7f22baf80980 20 add_watcher() i=2 2024-03-04T18:04:47.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.696+0000 7f22baf80980 20 add_watcher() i=3 2024-03-04T18:04:47.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.696+0000 7f22baf80980 20 add_watcher() i=4 2024-03-04T18:04:47.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.696+0000 7f22baf80980 20 add_watcher() i=5 2024-03-04T18:04:47.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.696+0000 7f22baf80980 20 add_watcher() i=6 2024-03-04T18:04:47.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.696+0000 7f22baf80980 20 add_watcher() i=7 2024-03-04T18:04:47.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.696+0000 7f22baf80980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:47.704 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.704+0000 7f216efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:47.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.704+0000 7f22baf80980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:47.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.704+0000 7f22baf80980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:47.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.704+0000 7f22baf80980 5 note: GC not initialized 2024-03-04T18:04:47.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.704+0000 7f216dfe3640 20 reqs_thread_entry: start 2024-03-04T18:04:47.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.704+0000 7f22baf80980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:47.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.704+0000 7f22baf80980 20 Filter name: none 2024-03-04T18:04:47.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.704+0000 7f20df7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:47.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.704+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.704+0000 7f22baf80980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:47.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.704+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.704+0000 7f22baf80980 20 rados_obj.operate() r=0 bl.length=425 2024-03-04T18:04:47.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.704+0000 7f22baf80980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:47.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.704+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.708+0000 7f22baf80980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:47.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.708+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.708+0000 7f22baf80980 20 rados_obj.operate() r=0 bl.length=425 2024-03-04T18:04:47.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.708+0000 7f22baf80980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:47.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.708+0000 7f22baf80980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.708+0000 7f22baf80980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.712 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.712+0000 7f216efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:47.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.716+0000 7f22baf80980 20 remove_watcher() i=0 2024-03-04T18:04:47.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.716+0000 7f22baf80980 2 removed watcher, disabling cache 2024-03-04T18:04:47.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.716+0000 7f22baf80980 20 remove_watcher() i=1 2024-03-04T18:04:47.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.720+0000 7f22baf80980 20 remove_watcher() i=2 2024-03-04T18:04:47.723 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.720+0000 7f22baf80980 20 remove_watcher() i=3 2024-03-04T18:04:47.725 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.724+0000 7f22baf80980 20 remove_watcher() i=4 2024-03-04T18:04:47.727 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.724+0000 7f22baf80980 20 remove_watcher() i=5 2024-03-04T18:04:47.730 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.728+0000 7f22baf80980 20 remove_watcher() i=6 2024-03-04T18:04:47.733 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.732+0000 7f22baf80980 20 remove_watcher() i=7 2024-03-04T18:04:47.749 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:47.749 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:04:47.749 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-commit-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:04:47.749 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:47.750 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:47.750 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:47.788 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:47.788 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:47.850 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:04:47.885 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:47.885 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:47.927 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.924+0000 7f14c8f52980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.928+0000 7f14c8f52980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.928+0000 7f14c8f52980 20 realm 2024-03-04T18:04:47.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.928+0000 7f14c8f52980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.928+0000 7f14c8f52980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.928+0000 7f14c8f52980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:47.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.928+0000 7f14c8f52980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.928+0000 7f14c8f52980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:47.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.928+0000 7f14c8f52980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.928+0000 7f14c8f52980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:47.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.928+0000 7f14c8f52980 20 rados->read ofs=0 len=0 2024-03-04T18:04:47.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.928+0000 7f14c8f52980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:47.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.928+0000 7f14c8f52980 20 searching for the correct realm 2024-03-04T18:04:47.936 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:47.936+0000 7f14c726b640 0 -- 172.21.15.137:0/3131435089 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f14c0084e40 msgr2=0x7f14c006ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:48.153 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.152+0000 7f14c726b640 0 -- 172.21.15.137:0/3131435089 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55a326f36640 msgr2=0x55a326f38b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:48.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:48.161 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:48.162 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:48.162 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:48.162 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:48.162 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:48.162 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.162 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:48.162 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.162 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:48.163 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:48.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 zone default found 2024-03-04T18:04:48.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 4 Realm: () 2024-03-04T18:04:48.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:48.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:48.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:48.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 zonegroup default 2024-03-04T18:04:48.164 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.160+0000 7f14c8f52980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.164+0000 7f14c8f52980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:48.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.164+0000 7f14c8f52980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.164+0000 7f14c8f52980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:48.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.164+0000 7f14c8f52980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.164+0000 7f14c8f52980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:48.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.164+0000 7f14c8f52980 20 started sync module instance, tier type = 2024-03-04T18:04:48.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.164+0000 7f14c8f52980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:48.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.184+0000 7f14c8f52980 20 add_watcher() i=0 2024-03-04T18:04:48.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.184+0000 7f14c8f52980 20 add_watcher() i=1 2024-03-04T18:04:48.186 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.184+0000 7f14c8f52980 20 add_watcher() i=2 2024-03-04T18:04:48.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.184+0000 7f14c8f52980 20 add_watcher() i=3 2024-03-04T18:04:48.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.184+0000 7f14c8f52980 20 add_watcher() i=4 2024-03-04T18:04:48.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.184+0000 7f14c8f52980 20 add_watcher() i=5 2024-03-04T18:04:48.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.184+0000 7f14c8f52980 20 add_watcher() i=6 2024-03-04T18:04:48.189 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.188+0000 7f14c8f52980 20 add_watcher() i=7 2024-03-04T18:04:48.189 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.188+0000 7f14c8f52980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:48.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.192+0000 7f137d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:48.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.192+0000 7f14c8f52980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:48.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.192+0000 7f14c8f52980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:48.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.192+0000 7f14c8f52980 5 note: GC not initialized 2024-03-04T18:04:48.196 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f12f7fff640 20 reqs_thread_entry: start 2024-03-04T18:04:48.196 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:48.196 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 20 Filter name: none 2024-03-04T18:04:48.197 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f12f5ffb640 20 reqs_thread_entry: start 2024-03-04T18:04:48.197 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-03-04T18:04:48.197 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:48.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-03-04T18:04:48.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-03-04T18:04:48.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 : miss 2024-03-04T18:04:48.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.199 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 20 rados_obj.operate() r=0 bl.length=425 2024-03-04T18:04:48.199 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 info.flags=0x17 2024-03-04T18:04:48.199 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 to cache LRU end 2024-03-04T18:04:48.199 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:48.199 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:48.199 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.196+0000 7f14c8f52980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 2024-03-04T18:04:48.201 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.200+0000 7f14c8f52980 10 init_target_layout removing existing target index objects from a previous reshard attempt 2024-03-04T18:04:48.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.208+0000 7f14c8f52980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 info.flags=0x17 2024-03-04T18:04:48.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.208+0000 7f14c8f52980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 to cache LRU end 2024-03-04T18:04:48.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.208+0000 7f14c8f52980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:48.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.208+0000 7f14c8f52980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25, ofs0, ns] 2024-03-04T18:04:48.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.208+0000 7f13aa7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627844 cookie 94159221522816 notifier 6296 bl.length()=965 2024-03-04T18:04:48.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.208+0000 7f13aa7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 info.flags=0x17 2024-03-04T18:04:48.210 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.208+0000 7f13aa7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 to cache LRU end 2024-03-04T18:04:48.210 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.208+0000 7f13aa7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:48.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.208+0000 7f14c8f52980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25]): orig_sources= new_sources= 2024-03-04T18:04:48.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.208+0000 7f14c8f52980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25]): potential sources added=[] removed=[] 2024-03-04T18:04:48.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.208+0000 7f14c8f52980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25]): orig_dests= new_dests= 2024-03-04T18:04:48.212 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.208+0000 7f14c8f52980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25]): potential dests added=[] removed=[] 2024-03-04T18:04:48.215 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.212+0000 7f14c8f52980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25.2.0 2024-03-04T18:04:48.215 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.212+0000 7f14c8f52980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25.2.0 2024-03-04T18:04:48.215 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.212+0000 7f14c8f52980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25.2.1 2024-03-04T18:04:48.215 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.212+0000 7f14c8f52980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25.2.1 2024-03-04T18:04:48.215 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.212+0000 7f14c8f52980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25.0 2024-03-04T18:04:48.221 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.220+0000 7f14c8f52980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 : hit (requested=0x17, cached=0x17) 2024-03-04T18:04:48.221 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.220+0000 7f14c8f52980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 2024-03-04T18:04:48.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.220+0000 7f14c8f52980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 info.flags=0x17 2024-03-04T18:04:48.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.220+0000 7f14c8f52980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 to cache LRU end 2024-03-04T18:04:48.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.220+0000 7f14c8f52980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:48.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.220+0000 7f14c8f52980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25, ofs0, ns] 2024-03-04T18:04:48.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.224+0000 7f13a9ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627845 cookie 94159221522816 notifier 6296 bl.length()=933 2024-03-04T18:04:48.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.224+0000 7f13a9ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 info.flags=0x17 2024-03-04T18:04:48.224 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.224+0000 7f13a9ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 to cache LRU end 2024-03-04T18:04:48.225 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.224+0000 7f13a9ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:48.226 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.224+0000 7f14c8f52980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25]): orig_sources= new_sources= 2024-03-04T18:04:48.226 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.224+0000 7f14c8f52980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25]): potential sources added=[] removed=[] 2024-03-04T18:04:48.226 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.224+0000 7f14c8f52980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25]): orig_dests= new_dests= 2024-03-04T18:04:48.226 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.224+0000 7f14c8f52980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25]): potential dests added=[] removed=[] 2024-03-04T18:04:48.229 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.228+0000 7f14c8f52980 1 execute INFO: reshard of bucket "error-at-commit-target-layout" completed successfully 2024-03-04T18:04:48.233 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.232+0000 7f137d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:48.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.236+0000 7f14c8f52980 20 remove_watcher() i=0 2024-03-04T18:04:48.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.236+0000 7f14c8f52980 2 removed watcher, disabling cache 2024-03-04T18:04:48.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.236+0000 7f14c8f52980 20 remove_watcher() i=1 2024-03-04T18:04:48.241 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.240+0000 7f14c8f52980 20 remove_watcher() i=2 2024-03-04T18:04:48.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.240+0000 7f14c8f52980 20 remove_watcher() i=3 2024-03-04T18:04:48.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.244+0000 7f14c8f52980 20 remove_watcher() i=4 2024-03-04T18:04:48.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.244+0000 7f14c8f52980 20 remove_watcher() i=5 2024-03-04T18:04:48.251 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.248+0000 7f14c8f52980 20 remove_watcher() i=6 2024-03-04T18:04:48.253 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.252+0000 7f14c8f52980 20 remove_watcher() i=7 2024-03-04T18:04:48.269 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-03-04T18:04:48.299 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:48.299 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:48.339 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.336+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.340+0000 7f4cc6dc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:48.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.340+0000 7f4cc6dc2980 20 realm 2024-03-04T18:04:48.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.340+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.340+0000 7f4cc6dc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:48.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.340+0000 7f4cc6dc2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:48.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.340+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.340+0000 7f4cc6dc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:48.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.340+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.340+0000 7f4cc6dc2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:48.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.340+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.344+0000 7f4cc6dc2980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:48.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.344+0000 7f4cc6dc2980 20 searching for the correct realm 2024-03-04T18:04:48.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.348+0000 7f4cc50db640 0 -- 172.21.15.137:0/2250679841 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f4cc00c8910 msgr2=0x7f4cc00d43f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:04:48.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.564+0000 7f4cc50db640 0 -- 172.21.15.137:0/2250679841 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x56487c1c9b70 msgr2=0x56487c1cc030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:48.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.572+0000 7f4cc6dc2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:48.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.572+0000 7f4cc6dc2980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:48.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.572+0000 7f4cc6dc2980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:48.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.572+0000 7f4cc6dc2980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:48.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.572+0000 7f4cc6dc2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:48.576 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.572+0000 7f4cc6dc2980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:48.576 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.572+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.576 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:48.576 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.577 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:48.577 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:48.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 20 zone default found 2024-03-04T18:04:48.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 4 Realm: () 2024-03-04T18:04:48.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:48.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:48.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:48.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 20 zonegroup default 2024-03-04T18:04:48.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:48.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:48.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.576+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.580 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.580+0000 7f4cc6dc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:48.580 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.580+0000 7f4cc6dc2980 20 started sync module instance, tier type = 2024-03-04T18:04:48.580 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.580+0000 7f4cc6dc2980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:48.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.600+0000 7f4cc6dc2980 20 add_watcher() i=0 2024-03-04T18:04:48.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.600+0000 7f4cc6dc2980 20 add_watcher() i=1 2024-03-04T18:04:48.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.600+0000 7f4cc6dc2980 20 add_watcher() i=2 2024-03-04T18:04:48.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.600+0000 7f4cc6dc2980 20 add_watcher() i=3 2024-03-04T18:04:48.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.600+0000 7f4cc6dc2980 20 add_watcher() i=4 2024-03-04T18:04:48.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.600+0000 7f4cc6dc2980 20 add_watcher() i=5 2024-03-04T18:04:48.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.600+0000 7f4cc6dc2980 20 add_watcher() i=6 2024-03-04T18:04:48.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.604+0000 7f4cc6dc2980 20 add_watcher() i=7 2024-03-04T18:04:48.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.604+0000 7f4cc6dc2980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:48.610 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.608+0000 7f4b7afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:48.611 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.608+0000 7f4cc6dc2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:48.611 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.608+0000 7f4cc6dc2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:48.611 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.608+0000 7f4cc6dc2980 5 note: GC not initialized 2024-03-04T18:04:48.611 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.608+0000 7f4b79fe3640 20 reqs_thread_entry: start 2024-03-04T18:04:48.611 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.608+0000 7f4cc6dc2980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:48.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.608+0000 7f4cc6dc2980 20 Filter name: none 2024-03-04T18:04:48.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.612+0000 7f4aeb7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:48.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.612+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.613 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.612+0000 7f4cc6dc2980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:48.613 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.612+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.614 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.612+0000 7f4cc6dc2980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:48.614 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.612+0000 7f4cc6dc2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:48.614 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.612+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.614 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.612+0000 7f4cc6dc2980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:48.614 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.612+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.612+0000 7f4cc6dc2980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:48.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.612+0000 7f4cc6dc2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:48.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.616+0000 7f4cc6dc2980 20 rados->read ofs=0 len=0 2024-03-04T18:04:48.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.616+0000 7f4cc6dc2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:48.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.616+0000 7f4b7afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:48.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.620+0000 7f4cc6dc2980 20 remove_watcher() i=0 2024-03-04T18:04:48.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.620+0000 7f4cc6dc2980 2 removed watcher, disabling cache 2024-03-04T18:04:48.624 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.624+0000 7f4cc6dc2980 20 remove_watcher() i=1 2024-03-04T18:04:48.626 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.624+0000 7f4cc6dc2980 20 remove_watcher() i=2 2024-03-04T18:04:48.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.628+0000 7f4cc6dc2980 20 remove_watcher() i=3 2024-03-04T18:04:48.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.628+0000 7f4cc6dc2980 20 remove_watcher() i=4 2024-03-04T18:04:48.633 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.632+0000 7f4cc6dc2980 20 remove_watcher() i=5 2024-03-04T18:04:48.635 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.632+0000 7f4cc6dc2980 20 remove_watcher() i=6 2024-03-04T18:04:48.638 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:48.636+0000 7f4cc6dc2980 20 remove_watcher() i=7 2024-03-04T18:04:48.659 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:48.659 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:04:48.659 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-commit-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.25 num_objects: 19 size_kb: 1 num_shards: 2 2024-03-04T18:04:48.659 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:48.660 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:48.660 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:48.679 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:48.679 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:49.078 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:49.078 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:49.249 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: reshard bucket with ECANCELED injected at commit_target_layout 2024-03-04T18:04:49.249 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:49.250 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:49.250 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:49.360 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:49.360 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:49.443 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:49.443 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:49.526 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:49.526 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:49.609 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:49.610 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:49.693 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:49.693 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:49.776 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:49.776 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:49.856 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:49.856 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:49.940 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:49.940 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:50.028 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:50.028 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:50.114 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:50.114 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:50.197 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:50.197 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:50.277 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:50.277 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:50.360 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:50.360 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:50.443 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:50.443 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:50.526 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:50.526 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:50.609 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:50.609 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:50.692 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:50.692 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:50.784 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:50.784 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:50.859 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:50.859 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:50.942 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:50.942 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:51.029 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:51.029 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:51.163 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-03-04T18:04:51.196 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:51.196 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:51.240 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.240+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.242 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.240+0000 7fa840e26980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.240+0000 7fa840e26980 20 realm 2024-03-04T18:04:51.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.240+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.240+0000 7fa840e26980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.240+0000 7fa840e26980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:51.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.240+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.240+0000 7fa840e26980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.243 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.240+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.244 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.244+0000 7fa840e26980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:51.244 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.244+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.244+0000 7fa840e26980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:51.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.244+0000 7fa840e26980 20 searching for the correct realm 2024-03-04T18:04:51.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.248+0000 7fa83f13f640 0 -- 172.21.15.137:0/2918093170 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x55e5ece2bfc0 msgr2=0x55e5ece2e480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:51.253 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.252+0000 7fa83e93e640 0 -- 172.21.15.137:0/2918093170 >> [v2:172.21.15.137:6824/1039931256,v1:172.21.15.137:6825/1039931256] conn(0x55e5ece30280 msgr2=0x55e5ece32670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:51.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:51.264 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:51.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:51.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:51.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:51.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:51.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:51.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:51.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 zone default found 2024-03-04T18:04:51.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 4 Realm: () 2024-03-04T18:04:51.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:51.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:51.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:51.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 zonegroup default 2024-03-04T18:04:51.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 started sync module instance, tier type = 2024-03-04T18:04:51.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.264+0000 7fa840e26980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:51.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.284+0000 7fa840e26980 20 add_watcher() i=0 2024-03-04T18:04:51.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.284+0000 7fa840e26980 20 add_watcher() i=1 2024-03-04T18:04:51.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.284+0000 7fa840e26980 20 add_watcher() i=2 2024-03-04T18:04:51.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.284+0000 7fa840e26980 20 add_watcher() i=3 2024-03-04T18:04:51.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.284+0000 7fa840e26980 20 add_watcher() i=4 2024-03-04T18:04:51.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.284+0000 7fa840e26980 20 add_watcher() i=5 2024-03-04T18:04:51.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.284+0000 7fa840e26980 20 add_watcher() i=6 2024-03-04T18:04:51.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.288+0000 7fa840e26980 20 add_watcher() i=7 2024-03-04T18:04:51.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.288+0000 7fa840e26980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:51.292 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa6f57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:51.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa840e26980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:51.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa840e26980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:51.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa840e26980 5 note: GC not initialized 2024-03-04T18:04:51.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa66ffff640 20 reqs_thread_entry: start 2024-03-04T18:04:51.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa840e26980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:51.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa840e26980 20 Filter name: none 2024-03-04T18:04:51.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa66dffb640 20 reqs_thread_entry: start 2024-03-04T18:04:51.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa840e26980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:51.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa840e26980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:51.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa840e26980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:51.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa840e26980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:51.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.292+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.296+0000 7fa840e26980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:51.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.296+0000 7fa840e26980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:51.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.296+0000 7fa840e26980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.296+0000 7fa840e26980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.296+0000 7fa6f57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:51.303 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.300+0000 7fa840e26980 20 remove_watcher() i=0 2024-03-04T18:04:51.303 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.300+0000 7fa840e26980 2 removed watcher, disabling cache 2024-03-04T18:04:51.305 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.304+0000 7fa840e26980 20 remove_watcher() i=1 2024-03-04T18:04:51.306 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.304+0000 7fa840e26980 20 remove_watcher() i=2 2024-03-04T18:04:51.308 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.308+0000 7fa840e26980 20 remove_watcher() i=3 2024-03-04T18:04:51.310 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.308+0000 7fa840e26980 20 remove_watcher() i=4 2024-03-04T18:04:51.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.312+0000 7fa840e26980 20 remove_watcher() i=5 2024-03-04T18:04:51.315 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.312+0000 7fa840e26980 20 remove_watcher() i=6 2024-03-04T18:04:51.318 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.316+0000 7fa840e26980 20 remove_watcher() i=7 2024-03-04T18:04:51.332 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:51.332 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:04:51.332 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-commit-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:04:51.332 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:51.333 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout --inject-error-code 125 2024-03-04T18:04:51.367 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:51.367 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:51.408 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.408+0000 7fd8de1ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.410 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.408+0000 7fd8de1ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.410 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.408+0000 7fd8de1ac980 20 realm 2024-03-04T18:04:51.410 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.408+0000 7fd8de1ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.408+0000 7fd8de1ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.408+0000 7fd8de1ac980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:51.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.408+0000 7fd8de1ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.408+0000 7fd8de1ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.411 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.408+0000 7fd8de1ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.412 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.412+0000 7fd8de1ac980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:51.412 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.412+0000 7fd8de1ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.413 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.412+0000 7fd8de1ac980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:51.413 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.412+0000 7fd8de1ac980 20 searching for the correct realm 2024-03-04T18:04:51.420 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.420+0000 7fd8db4c3640 0 -- 172.21.15.137:0/384231602 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7fd8cc028070 msgr2=0x7fd8cc014740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:51.427 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.424+0000 7fd8db4c3640 0 -- 172.21.15.137:0/384231602 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x56289f2b9a00 msgr2=0x56289f2bbe10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:51.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:51.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:51.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:51.432 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:51.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:51.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:51.433 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:51.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:51.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 zone default found 2024-03-04T18:04:51.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 4 Realm: () 2024-03-04T18:04:51.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:51.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:51.434 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:51.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 zonegroup default 2024-03-04T18:04:51.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.436 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.436 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.436 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 started sync module instance, tier type = 2024-03-04T18:04:51.436 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.432+0000 7fd8de1ac980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:51.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.452+0000 7fd8de1ac980 20 add_watcher() i=0 2024-03-04T18:04:51.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.452+0000 7fd8de1ac980 20 add_watcher() i=1 2024-03-04T18:04:51.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.452+0000 7fd8de1ac980 20 add_watcher() i=2 2024-03-04T18:04:51.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.452+0000 7fd8de1ac980 20 add_watcher() i=3 2024-03-04T18:04:51.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.452+0000 7fd8de1ac980 20 add_watcher() i=4 2024-03-04T18:04:51.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.452+0000 7fd8de1ac980 20 add_watcher() i=5 2024-03-04T18:04:51.453 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.452+0000 7fd8de1ac980 20 add_watcher() i=6 2024-03-04T18:04:51.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.452+0000 7fd8de1ac980 20 add_watcher() i=7 2024-03-04T18:04:51.455 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.452+0000 7fd8de1ac980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:51.459 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.456+0000 7fd791fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:51.459 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.456+0000 7fd8de1ac980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:51.459 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.456+0000 7fd8de1ac980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:51.459 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.456+0000 7fd8de1ac980 5 note: GC not initialized 2024-03-04T18:04:51.460 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.456+0000 7fd790fe1640 20 reqs_thread_entry: start 2024-03-04T18:04:51.460 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.456+0000 7fd8de1ac980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:51.460 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.456+0000 7fd8de1ac980 20 Filter name: none 2024-03-04T18:04:51.460 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.460+0000 7fd8de1ac980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-03-04T18:04:51.460 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.460+0000 7fd70a7fc640 20 reqs_thread_entry: start 2024-03-04T18:04:51.460 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.460+0000 7fd8de1ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.460+0000 7fd8de1ac980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:51.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.460+0000 7fd8de1ac980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-03-04T18:04:51.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.460+0000 7fd8de1ac980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-03-04T18:04:51.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.460+0000 7fd8de1ac980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 : miss 2024-03-04T18:04:51.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.460+0000 7fd8de1ac980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.462 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.460+0000 7fd8de1ac980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:51.462 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.460+0000 7fd8de1ac980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 info.flags=0x17 2024-03-04T18:04:51.462 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.460+0000 7fd8de1ac980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 to cache LRU end 2024-03-04T18:04:51.462 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.460+0000 7fd8de1ac980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:51.462 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.460+0000 7fd8de1ac980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:51.462 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.460+0000 7fd8de1ac980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 2024-03-04T18:04:51.469 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.468+0000 7fd8de1ac980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 info.flags=0x17 2024-03-04T18:04:51.469 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.468+0000 7fd8de1ac980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 to cache LRU end 2024-03-04T18:04:51.469 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.468+0000 7fd8de1ac980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:51.469 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.468+0000 7fd8de1ac980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26, ofs0, ns] 2024-03-04T18:04:51.469 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.468+0000 7fd7baffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627848 cookie 94732469120848 notifier 6347 bl.length()=965 2024-03-04T18:04:51.469 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.468+0000 7fd7baffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 info.flags=0x17 2024-03-04T18:04:51.469 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.468+0000 7fd7baffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 to cache LRU end 2024-03-04T18:04:51.469 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.468+0000 7fd7baffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:51.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.468+0000 7fd8de1ac980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26]): orig_sources= new_sources= 2024-03-04T18:04:51.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.468+0000 7fd8de1ac980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26]): potential sources added=[] removed=[] 2024-03-04T18:04:51.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.468+0000 7fd8de1ac980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26]): orig_dests= new_dests= 2024-03-04T18:04:51.471 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.468+0000 7fd8de1ac980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26]): potential dests added=[] removed=[] 2024-03-04T18:04:51.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.472+0000 7fd8de1ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26.1.0 2024-03-04T18:04:51.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.472+0000 7fd8de1ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26.1.0 2024-03-04T18:04:51.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.472+0000 7fd8de1ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26.1.1 2024-03-04T18:04:51.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.472+0000 7fd8de1ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26.1.1 2024-03-04T18:04:51.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.472+0000 7fd8de1ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26.0 2024-03-04T18:04:51.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.476+0000 7fd8de1ac980 -1 Injecting error=-125 at location=commit_target_layout 2024-03-04T18:04:51.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.476+0000 7fd8de1ac980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 : hit (requested=0x17, cached=0x17) 2024-03-04T18:04:51.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.476+0000 7fd8de1ac980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 2024-03-04T18:04:51.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.480+0000 7fd8de1ac980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 info.flags=0x17 2024-03-04T18:04:51.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.480+0000 7fd8de1ac980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 to cache LRU end 2024-03-04T18:04:51.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.480+0000 7fd8de1ac980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:51.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.480+0000 7fd8de1ac980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26, ofs0, ns] 2024-03-04T18:04:51.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.480+0000 7fd7baffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627849 cookie 94732469120848 notifier 6347 bl.length()=933 2024-03-04T18:04:51.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.480+0000 7fd7baffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 info.flags=0x17 2024-03-04T18:04:51.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.480+0000 7fd7baffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 to cache LRU end 2024-03-04T18:04:51.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.480+0000 7fd7baffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:51.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.480+0000 7fd8de1ac980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26]): orig_sources= new_sources= 2024-03-04T18:04:51.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.480+0000 7fd8de1ac980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26]): potential sources added=[] removed=[] 2024-03-04T18:04:51.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.480+0000 7fd8de1ac980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26]): orig_dests= new_dests= 2024-03-04T18:04:51.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.480+0000 7fd8de1ac980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26]): potential dests added=[] removed=[] 2024-03-04T18:04:51.486 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.484+0000 7fd8de1ac980 1 execute INFO: reshard of bucket "error-at-commit-target-layout" completed successfully 2024-03-04T18:04:51.488 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.488+0000 7fd791fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:51.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.492+0000 7fd8de1ac980 20 remove_watcher() i=0 2024-03-04T18:04:51.492 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.492+0000 7fd8de1ac980 2 removed watcher, disabling cache 2024-03-04T18:04:51.494 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.492+0000 7fd8de1ac980 20 remove_watcher() i=1 2024-03-04T18:04:51.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.496+0000 7fd8de1ac980 20 remove_watcher() i=2 2024-03-04T18:04:51.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.496+0000 7fd8de1ac980 20 remove_watcher() i=3 2024-03-04T18:04:51.500 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.500+0000 7fd8de1ac980 20 remove_watcher() i=4 2024-03-04T18:04:51.503 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.500+0000 7fd8de1ac980 20 remove_watcher() i=5 2024-03-04T18:04:51.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.504+0000 7fd8de1ac980 20 remove_watcher() i=6 2024-03-04T18:04:51.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.508+0000 7fd8de1ac980 20 remove_watcher() i=7 2024-03-04T18:04:51.522 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-03-04T18:04:51.549 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:51.549 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:51.792 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.792+0000 7fda6c1f7640 0 --2- 172.21.15.137:0/2662370261 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55bd25851c10 0x55bd2585c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-03-04T18:04:51.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.796+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.799 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.796+0000 7fda6eee0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.799 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.796+0000 7fda6eee0980 20 realm 2024-03-04T18:04:51.799 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.796+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.796+0000 7fda6eee0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.796+0000 7fda6eee0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:51.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.796+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.800+0000 7fda6eee0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:51.800 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.800+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.801 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.800+0000 7fda6eee0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:51.801 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.800+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:51.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.800+0000 7fda6eee0980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:51.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.800+0000 7fda6eee0980 20 searching for the correct realm 2024-03-04T18:04:51.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:51.804+0000 7fda6c1f7640 0 -- 172.21.15.137:0/2662370261 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7fda5c0de9e0 msgr2=0x7fda5c0bbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:52.019 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.016+0000 7fda6c1f7640 0 -- 172.21.15.137:0/2662370261 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x55bd2586c6f0 msgr2=0x55bd2586eae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:52.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.032+0000 7fda6eee0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:52.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.032+0000 7fda6eee0980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:52.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.032+0000 7fda6eee0980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:52.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.032+0000 7fda6eee0980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:52.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.032+0000 7fda6eee0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:52.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.032+0000 7fda6eee0980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:52.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.032+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:52.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:52.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:52.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:52.037 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:52.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:52.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 20 zone default found 2024-03-04T18:04:52.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 4 Realm: () 2024-03-04T18:04:52.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:52.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:52.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:52.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 20 zonegroup default 2024-03-04T18:04:52.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:52.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:52.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:52.039 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:52.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.036+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:52.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.040+0000 7fda6eee0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:52.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.040+0000 7fda6eee0980 20 started sync module instance, tier type = 2024-03-04T18:04:52.041 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.040+0000 7fda6eee0980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:52.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.064+0000 7fda6eee0980 20 add_watcher() i=0 2024-03-04T18:04:52.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.064+0000 7fda6eee0980 20 add_watcher() i=1 2024-03-04T18:04:52.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.064+0000 7fda6eee0980 20 add_watcher() i=2 2024-03-04T18:04:52.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.064+0000 7fda6eee0980 20 add_watcher() i=3 2024-03-04T18:04:52.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.064+0000 7fda6eee0980 20 add_watcher() i=4 2024-03-04T18:04:52.066 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.064+0000 7fda6eee0980 20 add_watcher() i=5 2024-03-04T18:04:52.066 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.064+0000 7fda6eee0980 20 add_watcher() i=6 2024-03-04T18:04:52.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.064+0000 7fda6eee0980 20 add_watcher() i=7 2024-03-04T18:04:52.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.064+0000 7fda6eee0980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:52.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.072+0000 7fd9237e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:52.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.072+0000 7fda6eee0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:52.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.072+0000 7fda6eee0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:52.073 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.072+0000 7fda6eee0980 5 note: GC not initialized 2024-03-04T18:04:52.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.072+0000 7fd9227e4640 20 reqs_thread_entry: start 2024-03-04T18:04:52.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.072+0000 7fda6eee0980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:52.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.072+0000 7fda6eee0980 20 Filter name: none 2024-03-04T18:04:52.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.072+0000 7fd91b7fe640 20 reqs_thread_entry: start 2024-03-04T18:04:52.075 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.072+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:52.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.072+0000 7fda6eee0980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:52.076 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.072+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:52.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.076+0000 7fda6eee0980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:52.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.076+0000 7fda6eee0980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:52.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.076+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:52.078 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.076+0000 7fda6eee0980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:52.078 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.076+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:52.080 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.076+0000 7fda6eee0980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:52.080 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.076+0000 7fda6eee0980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:52.081 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.080+0000 7fda6eee0980 20 rados->read ofs=0 len=0 2024-03-04T18:04:52.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.080+0000 7fda6eee0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:52.083 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.080+0000 7fd9237e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:52.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.084+0000 7fda6eee0980 20 remove_watcher() i=0 2024-03-04T18:04:52.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.084+0000 7fda6eee0980 2 removed watcher, disabling cache 2024-03-04T18:04:52.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.088+0000 7fda6eee0980 20 remove_watcher() i=1 2024-03-04T18:04:52.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.088+0000 7fda6eee0980 20 remove_watcher() i=2 2024-03-04T18:04:52.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.092+0000 7fda6eee0980 20 remove_watcher() i=3 2024-03-04T18:04:52.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.096+0000 7fda6eee0980 20 remove_watcher() i=4 2024-03-04T18:04:52.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.096+0000 7fda6eee0980 20 remove_watcher() i=5 2024-03-04T18:04:52.101 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.100+0000 7fda6eee0980 20 remove_watcher() i=6 2024-03-04T18:04:52.104 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:52.104+0000 7fda6eee0980 20 remove_watcher() i=7 2024-03-04T18:04:52.120 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:52.120 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:04:52.120 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-commit-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.26 num_objects: 20 size_kb: 1 num_shards: 2 2024-03-04T18:04:52.120 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:52.121 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:52.121 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:52.139 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:52.139 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:52.519 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:52.519 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:52.693 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: reshard bucket with abort at commit_target_layout 2024-03-04T18:04:52.693 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:52.698 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:52.698 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:52.807 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:52.807 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:52.888 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:52.888 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:52.974 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:52.974 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:53.057 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:53.057 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:53.141 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:53.141 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:53.223 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:53.223 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:53.307 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:53.307 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:53.393 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:53.393 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:53.476 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:53.476 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:53.560 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:53.560 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:53.644 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:53.644 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:53.726 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:53.727 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:53.811 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:53.811 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:53.896 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:53.896 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:53.981 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:53.981 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:54.066 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:54.067 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:54.150 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:54.151 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:54.234 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:54.234 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:54.317 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:54.317 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:54.401 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:54.401 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:54.485 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:54.485 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:04:54.567 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-03-04T18:04:54.599 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:54.599 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:54.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.640+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.644+0000 7f81cfad6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:54.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.644+0000 7f81cfad6980 20 realm 2024-03-04T18:04:54.645 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.644+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.646 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.644+0000 7f81cfad6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:54.646 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.644+0000 7f81cfad6980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:54.646 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.644+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.646 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.644+0000 7f81cfad6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:54.646 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.644+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.647 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.644+0000 7f81cfad6980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:54.647 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.644+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.648 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.648+0000 7f81cfad6980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:54.648 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.648+0000 7f81cfad6980 20 searching for the correct realm 2024-03-04T18:04:54.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.648+0000 7f81cd5ee640 0 -- 172.21.15.137:0/1276259345 >> [v2:172.21.15.137:6808/3491386063,v1:172.21.15.137:6810/3491386063] conn(0x55ab0ab4d770 msgr2=0x7f81c80f1260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:54.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.656+0000 7f81cddef640 0 -- 172.21.15.137:0/1276259345 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x55ab0ad82810 msgr2=0x55ab0ad84c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:54.666 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.664+0000 7f81cfad6980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:54.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.664+0000 7f81cfad6980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:54.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.664+0000 7f81cfad6980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:54.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.664+0000 7f81cfad6980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:54.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.664+0000 7f81cfad6980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:54.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.664+0000 7f81cfad6980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:54.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.664+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.664+0000 7f81cfad6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:54.667 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.664+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.668 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.664+0000 7f81cfad6980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:54.668 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.664+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.668 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:54.668 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 20 zone default found 2024-03-04T18:04:54.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 4 Realm: () 2024-03-04T18:04:54.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:54.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:54.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:54.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 20 zonegroup default 2024-03-04T18:04:54.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:54.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:54.669 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:54.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 20 started sync module instance, tier type = 2024-03-04T18:04:54.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.668+0000 7f81cfad6980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:54.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.688+0000 7f81cfad6980 20 add_watcher() i=0 2024-03-04T18:04:54.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.688+0000 7f81cfad6980 20 add_watcher() i=1 2024-03-04T18:04:54.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.688+0000 7f81cfad6980 20 add_watcher() i=2 2024-03-04T18:04:54.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.688+0000 7f81cfad6980 20 add_watcher() i=3 2024-03-04T18:04:54.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.688+0000 7f81cfad6980 20 add_watcher() i=4 2024-03-04T18:04:54.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.688+0000 7f81cfad6980 20 add_watcher() i=5 2024-03-04T18:04:54.688 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.688+0000 7f81cfad6980 20 add_watcher() i=6 2024-03-04T18:04:54.691 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.688+0000 7f81cfad6980 20 add_watcher() i=7 2024-03-04T18:04:54.691 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.688+0000 7f81cfad6980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:54.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.692+0000 7f80837e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:54.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.692+0000 7f81cfad6980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:54.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.692+0000 7f81cfad6980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:54.694 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.692+0000 7f81cfad6980 5 note: GC not initialized 2024-03-04T18:04:54.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.692+0000 7f80827e4640 20 reqs_thread_entry: start 2024-03-04T18:04:54.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.692+0000 7f81cfad6980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:54.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.692+0000 7f81cfad6980 20 Filter name: none 2024-03-04T18:04:54.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.692+0000 7f7ff3fff640 20 reqs_thread_entry: start 2024-03-04T18:04:54.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.692+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.696+0000 7f81cfad6980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:54.696 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.696+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.696+0000 7f81cfad6980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:54.697 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.696+0000 7f81cfad6980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:54.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.696+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.696+0000 7f81cfad6980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:54.698 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.696+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.696+0000 7f81cfad6980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:54.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.696+0000 7f81cfad6980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:54.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.696+0000 7f81cfad6980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.699 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.696+0000 7f81cfad6980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:54.700 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.700+0000 7f80837e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:54.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.704+0000 7f81cfad6980 20 remove_watcher() i=0 2024-03-04T18:04:54.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.704+0000 7f81cfad6980 2 removed watcher, disabling cache 2024-03-04T18:04:54.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.704+0000 7f81cfad6980 20 remove_watcher() i=1 2024-03-04T18:04:54.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.708+0000 7f81cfad6980 20 remove_watcher() i=2 2024-03-04T18:04:54.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.708+0000 7f81cfad6980 20 remove_watcher() i=3 2024-03-04T18:04:54.713 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.712+0000 7f81cfad6980 20 remove_watcher() i=4 2024-03-04T18:04:54.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.712+0000 7f81cfad6980 20 remove_watcher() i=5 2024-03-04T18:04:54.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.716+0000 7f81cfad6980 20 remove_watcher() i=6 2024-03-04T18:04:54.721 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.720+0000 7f81cfad6980 20 remove_watcher() i=7 2024-03-04T18:04:54.735 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:54.735 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:04:54.735 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: abort-at-commit-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:04:54.735 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:54.735 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at commit_target_layout 2024-03-04T18:04:54.767 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:54.767 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:54.810 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.808+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:54.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 realm 2024-03-04T18:04:54.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:54.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.812+0000 7fbd00f1c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:54.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:54.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.814 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:54.814 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:54.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:54.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 searching for the correct realm 2024-03-04T18:04:54.818 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:54.816+0000 7fbcfea34640 0 -- 172.21.15.137:0/1542649010 >> [v2:172.21.15.137:6808/3491386063,v1:172.21.15.137:6810/3491386063] conn(0x7fbcf8108070 msgr2=0x7fbcf80f1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:55.031 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.028+0000 7fbcff235640 0 -- 172.21.15.137:0/1542649010 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x557c67979e60 msgr2=0x557c6797c2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:55.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:55.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:55.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:55.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:55.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:55.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:55.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.048 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:55.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:55.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 zone default found 2024-03-04T18:04:55.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 4 Realm: () 2024-03-04T18:04:55.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:55.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:55.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:55.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 zonegroup default 2024-03-04T18:04:55.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.051 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.051 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.052 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.052+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.052 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.052+0000 7fbd00f1c980 20 started sync module instance, tier type = 2024-03-04T18:04:55.052 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.052+0000 7fbd00f1c980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:55.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=0 2024-03-04T18:04:55.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=1 2024-03-04T18:04:55.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=2 2024-03-04T18:04:55.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=3 2024-03-04T18:04:55.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=4 2024-03-04T18:04:55.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=5 2024-03-04T18:04:55.077 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=6 2024-03-04T18:04:55.080 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=7 2024-03-04T18:04:55.080 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.076+0000 7fbd00f1c980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:55.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbbb57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:55.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:55.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:55.085 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 note: GC not initialized 2024-03-04T18:04:55.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbb2ffff640 20 reqs_thread_entry: start 2024-03-04T18:04:55.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:55.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 Filter name: none 2024-03-04T18:04:55.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbb2dffb640 20 reqs_thread_entry: start 2024-03-04T18:04:55.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-03-04T18:04:55.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:55.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-03-04T18:04:55.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-03-04T18:04:55.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 : miss 2024-03-04T18:04:55.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:55.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 info.flags=0x17 2024-03-04T18:04:55.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 to cache LRU end 2024-03-04T18:04:55.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:55.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:55.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 2024-03-04T18:04:55.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.092+0000 7fbd00f1c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 info.flags=0x17 2024-03-04T18:04:55.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.092+0000 7fbd00f1c980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 to cache LRU end 2024-03-04T18:04:55.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.092+0000 7fbd00f1c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:55.095 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.092+0000 7fbd00f1c980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27, ofs0, ns] 2024-03-04T18:04:55.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.096+0000 7fbbe1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627822 cookie 93992802367728 notifier 6750 bl.length()=965 2024-03-04T18:04:55.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.096+0000 7fbbe1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 info.flags=0x17 2024-03-04T18:04:55.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.096+0000 7fbbe1ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 to cache LRU end 2024-03-04T18:04:55.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.096+0000 7fbbe1ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:55.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.096+0000 7fbd00f1c980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): orig_sources= new_sources= 2024-03-04T18:04:55.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.096+0000 7fbd00f1c980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): potential sources added=[] removed=[] 2024-03-04T18:04:55.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.096+0000 7fbd00f1c980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): orig_dests= new_dests= 2024-03-04T18:04:55.098 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.096+0000 7fbd00f1c980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): potential dests added=[] removed=[] 2024-03-04T18:04:55.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.100+0000 7fbd00f1c980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.0 2024-03-04T18:04:55.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.100+0000 7fbd00f1c980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.0 2024-03-04T18:04:55.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.100+0000 7fbd00f1c980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.1 2024-03-04T18:04:55.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.100+0000 7fbd00f1c980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.1 2024-03-04T18:04:55.100 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.100+0000 7fbd00f1c980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.0 2024-03-04T18:04:55.103 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fbd00f1c980 time 2024-03-04T18:04:55.107872+0000 2024-03-04T18:04:55.104 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:55.105 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:55.105 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fbd047f1d7b] 2024-03-04T18:04:55.105 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fbd047f1f2f] 2024-03-04T18:04:55.106 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557c64d7c281] 2024-03-04T18:04:55.106 INFO:tasks.workunit.client.0.smithi137.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x557c64d88fbf] 2024-03-04T18:04:55.106 INFO:tasks.workunit.client.0.smithi137.stderr: 5: main() 2024-03-04T18:04:55.106 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd0365cd90] 2024-03-04T18:04:55.106 INFO:tasks.workunit.client.0.smithi137.stderr: 7: __libc_start_main() 2024-03-04T18:04:55.106 INFO:tasks.workunit.client.0.smithi137.stderr: 8: _start() 2024-03-04T18:04:55.106 INFO:tasks.workunit.client.0.smithi137.stderr:*** Caught signal (Aborted) ** 2024-03-04T18:04:55.106 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7fbd00f1c980 thread_name:radosgw-admin 2024-03-04T18:04:55.106 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.104+0000 7fbd00f1c980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fbd00f1c980 time 2024-03-04T18:04:55.107872+0000 2024-03-04T18:04:55.106 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:55.106 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.106 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:55.106 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fbd047f1d7b] 2024-03-04T18:04:55.106 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fbd047f1f2f] 2024-03-04T18:04:55.107 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557c64d7c281] 2024-03-04T18:04:55.107 INFO:tasks.workunit.client.0.smithi137.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x557c64d88fbf] 2024-03-04T18:04:55.107 INFO:tasks.workunit.client.0.smithi137.stderr: 5: main() 2024-03-04T18:04:55.107 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd0365cd90] 2024-03-04T18:04:55.107 INFO:tasks.workunit.client.0.smithi137.stderr: 7: __libc_start_main() 2024-03-04T18:04:55.107 INFO:tasks.workunit.client.0.smithi137.stderr: 8: _start() 2024-03-04T18:04:55.107 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.132 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:55.132 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fbd03675520] 2024-03-04T18:04:55.133 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:04:55.133 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:04:55.133 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:04:55.133 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fbd047f1dd5] 2024-03-04T18:04:55.133 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fbd047f1f2f] 2024-03-04T18:04:55.133 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557c64d7c281] 2024-03-04T18:04:55.133 INFO:tasks.workunit.client.0.smithi137.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x557c64d88fbf] 2024-03-04T18:04:55.133 INFO:tasks.workunit.client.0.smithi137.stderr: 9: main() 2024-03-04T18:04:55.133 INFO:tasks.workunit.client.0.smithi137.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd0365cd90] 2024-03-04T18:04:55.133 INFO:tasks.workunit.client.0.smithi137.stderr: 11: __libc_start_main() 2024-03-04T18:04:55.133 INFO:tasks.workunit.client.0.smithi137.stderr: 12: _start() 2024-03-04T18:04:55.133 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.104+0000 7fbd00f1c980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:04:55.133 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7fbd00f1c980 thread_name:radosgw-admin 2024-03-04T18:04:55.133 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fbd03675520] 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fbd047f1dd5] 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fbd047f1f2f] 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557c64d7c281] 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x557c64d88fbf] 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: 9: main() 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd0365cd90] 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: 11: __libc_start_main() 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: 12: _start() 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.134 INFO:tasks.workunit.client.0.smithi137.stderr:--- begin dump of recent events --- 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr: -499> 2024-03-04T18:04:54.788+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr: -498> 2024-03-04T18:04:54.788+0000 7fbcfe233640 10 monclient(hunting): handle_auth_done global_id 6738 payload 274 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr: -497> 2024-03-04T18:04:54.788+0000 7fbcfe233640 10 monclient: _finish_hunting 0 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr: -496> 2024-03-04T18:04:54.788+0000 7fbcfe233640 1 monclient: found mon.noname-a 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr: -495> 2024-03-04T18:04:54.788+0000 7fbcfe233640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr: -494> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr: -493> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr: -492> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: dump: 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.135 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr: -491> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: _finish_auth 0 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr: -490> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: _check_auth_tickets 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr: -489> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 monclient: authenticate success, global_id 6738 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr: -488> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr: -487> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr: -486> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr: -485> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: dump: 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.136 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr: -484> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr: -483> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 4 set_mon_vals no callback set 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr: -482> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 10 monclient: shutdown 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr: -481> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 asok(0x557c676ad130) unregister_commands rotate-key 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr: -480> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 10 monclient: build_initial_monmap 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr: -479> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 1 build_initial for_mkfs: 0 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr: -478> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 10 monclient: monmap: 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:55.137 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr: -477> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding auth protocol: cephx 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr: -476> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding auth protocol: cephx 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr: -475> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding auth protocol: cephx 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr: -474> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: secure 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr: -473> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: crc 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr: -472> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: secure 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr: -471> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: crc 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr: -470> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: secure 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr: -469> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: crc 2024-03-04T18:04:55.138 INFO:tasks.workunit.client.0.smithi137.stderr: -468> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: crc 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -467> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: secure 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -466> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: crc 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -465> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: secure 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -464> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: crc 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -463> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: secure 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -462> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -461> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 1 librados: starting msgr at 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -460> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 1 librados: starting objecter 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -459> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command objecter_requests hook 0x557c6767df10 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -458> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 1 librados: setting wanted keys 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -457> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 1 librados: calling monclient init 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -456> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 10 monclient: init 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -455> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding auth protocol: cephx 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -454> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding auth protocol: cephx 2024-03-04T18:04:55.139 INFO:tasks.workunit.client.0.smithi137.stderr: -453> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding auth protocol: cephx 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -452> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: secure 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -451> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: crc 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -450> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: secure 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -449> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: crc 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -448> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: secure 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -447> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: crc 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -446> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: crc 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -445> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: secure 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -444> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: crc 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -443> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: secure 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -442> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: crc 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -441> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: secure 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -440> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -439> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -438> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command rotate-key hook 0x557c6788fad8 2024-03-04T18:04:55.140 INFO:tasks.workunit.client.0.smithi137.stderr: -437> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -436> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient: _add_conns ranks=[0,2,1] 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -435> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-a con 0x557c678961f0 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -434> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-c con 0x557c678be6c0 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -433> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-b con 0x7fbce8003ac0 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -432> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -431> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -430> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -429> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -428> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): authenticate will time out at 2527.912450s 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -427> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): get_auth_request con 0x557c678961f0 auth_method 0 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -426> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -425> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -424> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -423> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient(hunting): get_auth_request con 0x557c678be6c0 auth_method 0 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -422> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -421> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.141 INFO:tasks.workunit.client.0.smithi137.stderr: -420> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr: -419> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr: -418> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr: -417> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr: -416> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr: -415> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr: -414> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr: -413> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): handle_auth_done global_id 6741 payload 824 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr: -412> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient: _finish_hunting 0 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr: -411> 2024-03-04T18:04:54.792+0000 7fbcfea34640 1 monclient: found mon.noname-a 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr: -410> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr: -409> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr: -408> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr: -407> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: dump: 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.142 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr: -406> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: _finish_auth 0 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr: -405> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: _check_auth_tickets 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr: -404> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 5 monclient: authenticate success, global_id 6741 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr: -403> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient: _renew_subs 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr: -402> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr: -401> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr: -400> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr: -399> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr: -398> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: dump: 2024-03-04T18:04:55.143 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.144 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.144 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.144 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.144 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.144 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.144 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.144 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.144 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.144 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.144 INFO:tasks.workunit.client.0.smithi137.stderr: -397> 2024-03-04T18:04:54.792+0000 7fbbece09640 4 set_mon_vals no callback set 2024-03-04T18:04:55.144 INFO:tasks.workunit.client.0.smithi137.stderr: -396> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient: _renew_subs 2024-03-04T18:04:55.144 INFO:tasks.workunit.client.0.smithi137.stderr: -395> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.144 INFO:tasks.workunit.client.0.smithi137.stderr: -394> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 1 librados: init done 2024-03-04T18:04:55.144 INFO:tasks.workunit.client.0.smithi137.stderr: -393> 2024-03-04T18:04:54.792+0000 7fbbeb606640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr: -392> 2024-03-04T18:04:54.792+0000 7fbbeb606640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr: -391> 2024-03-04T18:04:54.792+0000 7fbbeb606640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr: -390> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient: get_auth_request con 0x7fbbe4065970 auth_method 0 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr: -389> 2024-03-04T18:04:54.792+0000 7fbcff235640 10 monclient: get_auth_request con 0x557c678d82d0 auth_method 0 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr: -388> 2024-03-04T18:04:54.796+0000 7fbcfea34640 10 monclient: get_auth_request con 0x557c678db110 auth_method 0 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr: -387> 2024-03-04T18:04:54.796+0000 7fbcfe233640 10 monclient: get_auth_request con 0x557c678df300 auth_method 0 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr: -386> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 10 monclient: build_initial_monmap 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr: -385> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 1 build_initial for_mkfs: 0 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr: -384> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 10 monclient: monmap: 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:55.145 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr: -383> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding auth protocol: cephx 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr: -382> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding auth protocol: cephx 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr: -381> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding auth protocol: cephx 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr: -380> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: secure 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr: -379> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: crc 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr: -378> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: secure 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr: -377> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: crc 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr: -376> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: secure 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr: -375> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: crc 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr: -374> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: crc 2024-03-04T18:04:55.146 INFO:tasks.workunit.client.0.smithi137.stderr: -373> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: secure 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -372> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: crc 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -371> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: secure 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -370> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: crc 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -369> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: secure 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -368> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -367> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command objecter_requests cmddesc objecter_requests hook 0x557c678e93f0 EEXIST 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -366> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: init 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -365> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding auth protocol: cephx 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -364> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding auth protocol: cephx 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -363> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding auth protocol: cephx 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -362> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: secure 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -361> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: crc 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -360> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: secure 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -359> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: crc 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -358> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: secure 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -357> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: crc 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -356> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: crc 2024-03-04T18:04:55.147 INFO:tasks.workunit.client.0.smithi137.stderr: -355> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: secure 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -354> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: crc 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -353> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: secure 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -352> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: crc 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -351> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: secure 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -350> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -349> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -348> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command rotate-key cmddesc rotate-key hook 0x557c678e2660 EEXIST 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -347> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -346> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: _add_conns ranks=[0,1,2] 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -345> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-a con 0x557c6790e050 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -344> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-b con 0x557c6790e430 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -343> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-c con 0x557c679095f0 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -342> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -341> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.148 INFO:tasks.workunit.client.0.smithi137.stderr: -340> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -339> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -338> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): authenticate will time out at 2527.920678s 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -337> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient(hunting): get_auth_request con 0x557c6790e050 auth_method 0 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -336> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -335> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -334> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -333> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): get_auth_request con 0x557c679095f0 auth_method 0 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -332> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -331> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -330> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -329> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -328> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -327> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -326> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -325> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.149 INFO:tasks.workunit.client.0.smithi137.stderr: -324> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr: -323> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): handle_auth_done global_id 6386 payload 824 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr: -322> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient: _finish_hunting 0 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr: -321> 2024-03-04T18:04:54.800+0000 7fbcff235640 1 monclient: found mon.noname-c 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr: -320> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.137:3301/0 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr: -319> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr: -318> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr: -317> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: dump: 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.150 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr: -316> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: _finish_auth 0 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr: -315> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: _check_auth_tickets 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr: -314> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr: -313> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr: -312> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr: -311> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: dump: 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.151 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr: -310> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 monclient: authenticate success, global_id 6386 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr: -309> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: _renew_subs 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr: -308> 2024-03-04T18:04:54.800+0000 7fbcffa36640 4 set_mon_vals no callback set 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr: -307> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.137:3301/0 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr: -306> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: _renew_subs 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr: -305> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.137:3301/0 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr: -304> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr: -303> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr: -302> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr: -301> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: get_monmap_and_config 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr: -300> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: build_initial_monmap 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr: -299> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 1 build_initial for_mkfs: 0 2024-03-04T18:04:55.152 INFO:tasks.workunit.client.0.smithi137.stderr: -298> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: monmap: 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr: -297> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding auth protocol: cephx 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr: -296> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding auth protocol: cephx 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr: -295> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding auth protocol: cephx 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr: -294> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr: -293> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.153 INFO:tasks.workunit.client.0.smithi137.stderr: -292> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -291> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -290> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -289> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -288> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -287> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -286> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -285> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -284> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -283> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -282> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient: get_auth_request con 0x7fbbd8065880 auth_method 0 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -281> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -280> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: init 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -279> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding auth protocol: cephx 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -278> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding auth protocol: cephx 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -277> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding auth protocol: cephx 2024-03-04T18:04:55.154 INFO:tasks.workunit.client.0.smithi137.stderr: -276> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: secure 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -275> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: crc 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -274> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: secure 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -273> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: crc 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -272> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: secure 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -271> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: crc 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -270> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: crc 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -269> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: secure 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -268> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: crc 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -267> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: secure 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -266> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: crc 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -265> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: secure 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -264> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -263> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -262> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command rotate-key cmddesc rotate-key hook 0x7ffec3269d38 EEXIST 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -261> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:55.155 INFO:tasks.workunit.client.0.smithi137.stderr: -260> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: _add_conns ranks=[2,1,0] 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -259> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-c con 0x557c679523b0 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -258> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-b con 0x557c6795ca70 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -257> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-a con 0x557c6792d0d0 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -256> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -255> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -254> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -253> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -252> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): authenticate will time out at 2527.924796s 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -251> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): get_auth_request con 0x557c6792d0d0 auth_method 0 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -250> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -249> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -248> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -247> 2024-03-04T18:04:54.804+0000 7fbcfea34640 10 monclient(hunting): get_auth_request con 0x557c679523b0 auth_method 0 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -246> 2024-03-04T18:04:54.804+0000 7fbcfea34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.156 INFO:tasks.workunit.client.0.smithi137.stderr: -245> 2024-03-04T18:04:54.804+0000 7fbcfea34640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr: -244> 2024-03-04T18:04:54.804+0000 7fbcfea34640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr: -243> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr: -242> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr: -241> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr: -240> 2024-03-04T18:04:54.804+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr: -239> 2024-03-04T18:04:54.804+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr: -238> 2024-03-04T18:04:54.804+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr: -237> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): handle_auth_done global_id 6747 payload 274 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr: -236> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient: _finish_hunting 0 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr: -235> 2024-03-04T18:04:54.804+0000 7fbcff235640 1 monclient: found mon.noname-a 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr: -234> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr: -233> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr: -232> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr: -231> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: dump: 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.157 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr: -230> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: _finish_auth 0 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr: -229> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: _check_auth_tickets 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr: -228> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr: -227> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr: -226> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr: -225> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: dump: 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.158 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr: -224> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 monclient: authenticate success, global_id 6747 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr: -223> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr: -222> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 4 set_mon_vals no callback set 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr: -221> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: shutdown 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr: -220> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: build_initial_monmap 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr: -219> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 1 build_initial for_mkfs: 0 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr: -218> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: monmap: 2024-03-04T18:04:55.159 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr: -217> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding auth protocol: cephx 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr: -216> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding auth protocol: cephx 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr: -215> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding auth protocol: cephx 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr: -214> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr: -213> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr: -212> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.160 INFO:tasks.workunit.client.0.smithi137.stderr: -211> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -210> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -209> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -208> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -207> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -206> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -205> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -204> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -203> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -202> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -201> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 1 librados: starting msgr at 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -200> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 1 librados: starting objecter 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -199> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command objecter_requests cmddesc objecter_requests hook 0x557c6795ca50 EEXIST 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -198> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 1 librados: setting wanted keys 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -197> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 1 librados: calling monclient init 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -196> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: init 2024-03-04T18:04:55.161 INFO:tasks.workunit.client.0.smithi137.stderr: -195> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding auth protocol: cephx 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -194> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding auth protocol: cephx 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -193> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding auth protocol: cephx 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -192> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: secure 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -191> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: crc 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -190> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: secure 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -189> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: crc 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -188> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: secure 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -187> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: crc 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -186> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: crc 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -185> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: secure 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -184> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: crc 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -183> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: secure 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -182> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: crc 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -181> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: secure 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -180> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -179> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.162 INFO:tasks.workunit.client.0.smithi137.stderr: -178> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command rotate-key cmddesc rotate-key hook 0x557c6792aeb8 EEXIST 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -177> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -176> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -175> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-b con 0x557c6792d0d0 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -174> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-a con 0x557c679523b0 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -173> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-c con 0x557c6795ca70 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -172> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -171> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -170> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -169> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -168> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): authenticate will time out at 2527.927381s 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -167> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): get_auth_request con 0x557c679523b0 auth_method 0 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -166> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -165> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -164> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.163 INFO:tasks.workunit.client.0.smithi137.stderr: -163> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): get_auth_request con 0x557c6795ca70 auth_method 0 2024-03-04T18:04:55.164 INFO:tasks.workunit.client.0.smithi137.stderr: -162> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.164 INFO:tasks.workunit.client.0.smithi137.stderr: -161> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.164 INFO:tasks.workunit.client.0.smithi137.stderr: -160> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.164 INFO:tasks.workunit.client.0.smithi137.stderr: -159> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.164 INFO:tasks.workunit.client.0.smithi137.stderr: -158> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.164 INFO:tasks.workunit.client.0.smithi137.stderr: -157> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.164 INFO:tasks.workunit.client.0.smithi137.stderr: -156> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.164 INFO:tasks.workunit.client.0.smithi137.stderr: -155> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.164 INFO:tasks.workunit.client.0.smithi137.stderr: -154> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.164 INFO:tasks.workunit.client.0.smithi137.stderr: -153> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): handle_auth_done global_id 6750 payload 824 2024-03-04T18:04:55.164 INFO:tasks.workunit.client.0.smithi137.stderr: -152> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient: _finish_hunting 0 2024-03-04T18:04:55.164 INFO:tasks.workunit.client.0.smithi137.stderr: -151> 2024-03-04T18:04:54.804+0000 7fbcfe233640 1 monclient: found mon.noname-a 2024-03-04T18:04:55.164 INFO:tasks.workunit.client.0.smithi137.stderr: -150> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.164 INFO:tasks.workunit.client.0.smithi137.stderr: -149> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr: -148> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr: -147> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: dump: 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr: -146> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: _finish_auth 0 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr: -145> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: _check_auth_tickets 2024-03-04T18:04:55.165 INFO:tasks.workunit.client.0.smithi137.stderr: -144> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr: -143> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr: -142> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr: -141> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: dump: 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr: -140> 2024-03-04T18:04:54.808+0000 7fbbe27fc640 4 set_mon_vals no callback set 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr: -139> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 5 monclient: authenticate success, global_id 6750 2024-03-04T18:04:55.166 INFO:tasks.workunit.client.0.smithi137.stderr: -138> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 10 monclient: _renew_subs 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -137> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -136> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 10 monclient: _renew_subs 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -135> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -134> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 1 librados: init done 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -133> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command cr dump hook 0x557c6795db58 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -132> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -131> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -130> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -129> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -128> 2024-03-04T18:04:54.808+0000 7fbcfea34640 10 monclient: get_auth_request con 0x7fbbc8065a30 auth_method 0 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -127> 2024-03-04T18:04:54.808+0000 7fbcff235640 10 monclient: get_auth_request con 0x557c67979e60 auth_method 0 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -126> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -125> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 realm 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -124> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -123> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -122> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:55.167 INFO:tasks.workunit.client.0.smithi137.stderr: -121> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -120> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -119> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -118> 2024-03-04T18:04:54.812+0000 7fbcfea34640 10 monclient: get_auth_request con 0x557c6797e0d0 auth_method 0 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -117> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -116> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -115> 2024-03-04T18:04:54.812+0000 7fbcfe233640 10 monclient: get_auth_request con 0x557c67981910 auth_method 0 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -114> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -113> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 searching for the correct realm 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -112> 2024-03-04T18:04:54.812+0000 7fbcff235640 10 monclient: get_auth_request con 0x557c67986610 auth_method 0 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -111> 2024-03-04T18:04:54.816+0000 7fbbe0ff9640 1 client.6750.objecter ms_handle_reset 0x557c67986610 session 0x557c67985f90 osd.4 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -110> 2024-03-04T18:04:54.816+0000 7fbbe0ff9640 10 monclient: _renew_subs 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -109> 2024-03-04T18:04:54.816+0000 7fbbe0ff9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -108> 2024-03-04T18:04:54.816+0000 7fbcff235640 10 monclient: get_auth_request con 0x7fbbc80754d0 auth_method 0 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -107> 2024-03-04T18:04:54.816+0000 7fbcfea34640 10 monclient: get_auth_request con 0x7fbcf8108070 auth_method 0 2024-03-04T18:04:55.168 INFO:tasks.workunit.client.0.smithi137.stderr: -106> 2024-03-04T18:04:54.816+0000 7fbcfea34640 0 -- 172.21.15.137:0/1542649010 >> [v2:172.21.15.137:6808/3491386063,v1:172.21.15.137:6810/3491386063] conn(0x7fbcf8108070 msgr2=0x7fbcf80f1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -105> 2024-03-04T18:04:55.020+0000 7fbcfea34640 10 monclient: get_auth_request con 0x7fbcf8108070 auth_method 0 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -104> 2024-03-04T18:04:55.024+0000 7fbcfe233640 10 monclient: get_auth_request con 0x7fbcf0010040 auth_method 0 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -103> 2024-03-04T18:04:55.024+0000 7fbcff235640 10 monclient: get_auth_request con 0x557c67744fd0 auth_method 0 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -102> 2024-03-04T18:04:55.028+0000 7fbcff235640 0 -- 172.21.15.137:0/1542649010 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x557c67979e60 msgr2=0x557c6797c2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -101> 2024-03-04T18:04:55.028+0000 7fbbe0ff9640 1 client.6750.objecter ms_handle_reset 0x557c67979e60 session 0x557c679797e0 osd.6 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -100> 2024-03-04T18:04:55.028+0000 7fbcff235640 10 monclient: get_auth_request con 0x557c67986610 auth_method 0 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -99> 2024-03-04T18:04:55.032+0000 7fbcfea34640 10 monclient: get_auth_request con 0x7fbcf0053470 auth_method 0 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -98> 2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -97> 2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -96> 2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -95> 2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -94> 2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -93> 2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -92> 2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -91> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -90> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.169 INFO:tasks.workunit.client.0.smithi137.stderr: -89> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -88> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -87> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -86> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 zone default found 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -85> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 4 Realm: () 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -84> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -83> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -82> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -81> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 zonegroup default 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -80> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -79> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -78> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -77> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -76> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -75> 2024-03-04T18:04:55.052+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.170 INFO:tasks.workunit.client.0.smithi137.stderr: -74> 2024-03-04T18:04:55.052+0000 7fbd00f1c980 20 started sync module instance, tier type = 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -73> 2024-03-04T18:04:55.052+0000 7fbd00f1c980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -72> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=0 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -71> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=1 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -70> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=2 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -69> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=3 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -68> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=4 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -67> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=5 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -66> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=6 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -65> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=7 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -64> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -63> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding auth protocol: cephx 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -62> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding auth protocol: cephx 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -61> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding auth protocol: cephx 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -60> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: secure 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -59> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: crc 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -58> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: secure 2024-03-04T18:04:55.171 INFO:tasks.workunit.client.0.smithi137.stderr: -57> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: crc 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -56> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: secure 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -55> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: crc 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -54> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: crc 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -53> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: secure 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -52> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: crc 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -51> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: secure 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -50> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: crc 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -49> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: secure 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -48> 2024-03-04T18:04:55.084+0000 7fbbb57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -47> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -46> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -45> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -44> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command cache list hook 0x557c6792d4d0 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -43> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command cache inspect hook 0x557c6792d4d0 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -42> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command cache erase hook 0x557c6792d4d0 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -41> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command cache zap hook 0x557c6792d4d0 2024-03-04T18:04:55.172 INFO:tasks.workunit.client.0.smithi137.stderr: -40> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 note: GC not initialized 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -39> 2024-03-04T18:04:55.084+0000 7fbb2ffff640 20 reqs_thread_entry: start 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -38> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command sync trace show hook 0x7fbcf8002170 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -37> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command sync trace history hook 0x7fbcf8002170 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -36> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command sync trace active hook 0x7fbcf8002170 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -35> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command sync trace active_short hook 0x7fbcf8002170 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -34> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -33> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 Filter name: none 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -32> 2024-03-04T18:04:55.084+0000 7fbb2dffb640 20 reqs_thread_entry: start 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -31> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -30> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -29> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -28> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -27> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -26> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 : miss 2024-03-04T18:04:55.173 INFO:tasks.workunit.client.0.smithi137.stderr: -25> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -24> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -23> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 info.flags=0x17 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -22> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 to cache LRU end 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -21> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -20> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -19> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -18> 2024-03-04T18:04:55.092+0000 7fbd00f1c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 info.flags=0x17 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -17> 2024-03-04T18:04:55.092+0000 7fbd00f1c980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 to cache LRU end 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -16> 2024-03-04T18:04:55.092+0000 7fbd00f1c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -15> 2024-03-04T18:04:55.092+0000 7fbd00f1c980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27, ofs0, ns] 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -14> 2024-03-04T18:04:55.096+0000 7fbbe1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627822 cookie 93992802367728 notifier 6750 bl.length()=965 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -13> 2024-03-04T18:04:55.096+0000 7fbbe1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 info.flags=0x17 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -12> 2024-03-04T18:04:55.096+0000 7fbbe1ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 to cache LRU end 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -11> 2024-03-04T18:04:55.096+0000 7fbbe1ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -10> 2024-03-04T18:04:55.096+0000 7fbd00f1c980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): orig_sources= new_sources= 2024-03-04T18:04:55.174 INFO:tasks.workunit.client.0.smithi137.stderr: -9> 2024-03-04T18:04:55.096+0000 7fbd00f1c980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): potential sources added=[] removed=[] 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: -8> 2024-03-04T18:04:55.096+0000 7fbd00f1c980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): orig_dests= new_dests= 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: -7> 2024-03-04T18:04:55.096+0000 7fbd00f1c980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): potential dests added=[] removed=[] 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: -6> 2024-03-04T18:04:55.100+0000 7fbd00f1c980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.0 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: -5> 2024-03-04T18:04:55.100+0000 7fbd00f1c980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.0 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: -4> 2024-03-04T18:04:55.100+0000 7fbd00f1c980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.1 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: -3> 2024-03-04T18:04:55.100+0000 7fbd00f1c980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.1 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: -2> 2024-03-04T18:04:55.100+0000 7fbd00f1c980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.0 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: -1> 2024-03-04T18:04:55.104+0000 7fbd00f1c980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fbd00f1c980 time 2024-03-04T18:04:55.107872+0000 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fbd047f1d7b] 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fbd047f1f2f] 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557c64d7c281] 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x557c64d88fbf] 2024-03-04T18:04:55.175 INFO:tasks.workunit.client.0.smithi137.stderr: 5: main() 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd0365cd90] 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: 7: __libc_start_main() 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: 8: _start() 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: 0> 2024-03-04T18:04:55.104+0000 7fbd00f1c980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7fbd00f1c980 thread_name:radosgw-admin 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fbd03675520] 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fbd047f1dd5] 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fbd047f1f2f] 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557c64d7c281] 2024-03-04T18:04:55.176 INFO:tasks.workunit.client.0.smithi137.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x557c64d88fbf] 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 9: main() 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd0365cd90] 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 11: __libc_start_main() 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 12: _start() 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr:--- logging levels --- 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 none 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 lockdep 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 context 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 crush 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_balancer 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_locker 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log_expire 2024-03-04T18:04:55.177 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_migrator 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 buffer 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 timer 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 filer 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 striper 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 objecter 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rados 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_mirror 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_replay 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_pwl 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 journaler 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objectcacher 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 client 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 osd 2024-03-04T18:04:55.178 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 optracker 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objclass 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 filestore 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 journal 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 ms 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mon 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 0/10 monc 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 paxos 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 tp 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 auth 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 crypto 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 finisher 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 reserver 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 heartbeatmap 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 perfcounter 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 20/20 rgw 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_sync 2024-03-04T18:04:55.179 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_datacache 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_access 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_flight 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 javaclient 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 asok 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 throttle 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 refs 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 compressor 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluestore 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluefs 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 bdev 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 kstore 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 4/ 5 rocksdb 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 fuse 2024-03-04T18:04:55.180 INFO:tasks.workunit.client.0.smithi137.stderr: 2/ 5 mgr 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mgrc 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 dpdk 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 eventtrace 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 prioritycache 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 test 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephsqlite 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_onode 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_odata 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_omap 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_tm 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_t 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:04:55.181 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_epm 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_lba 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cache 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_journal 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_device 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_backref 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 alienstore 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mclock 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cyanstore 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 ceph_exporter 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 memstore 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 trace 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: -2/-2 (syslog threshold) 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 99/99 (stderr threshold) 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbb2dffb640 / http_manager 2024-03-04T18:04:55.182 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbb2ffff640 / http_manager 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbbb57e2640 / rgw_dt_lg_renew 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbbe0ff9640 / ms_dispatch 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbbe1ffb640 / io_context_pool 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbbe27fc640 / io_context_pool 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbbe8e01640 / ms_dispatch 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbbeb606640 / ms_dispatch 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbbece09640 / io_context_pool 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbcfe233640 / msgr-worker-2 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbcfea34640 / msgr-worker-1 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbcff235640 / msgr-worker-0 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbcffa36640 / io_context_pool 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbd00f1c980 / radosgw-admin 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: max_recent 500 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: max_new 1000 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr: log_file /var/log/ceph/ceph-client.admin.95939.log 2024-03-04T18:04:55.183 INFO:tasks.workunit.client.0.smithi137.stderr:--- end dump of recent events --- 2024-03-04T18:04:55.184 INFO:tasks.workunit.client.0.smithi137.stderr:--- begin dump of recent events --- 2024-03-04T18:04:55.184 INFO:tasks.workunit.client.0.smithi137.stderr: -499> 2024-03-04T18:04:54.788+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.184 INFO:tasks.workunit.client.0.smithi137.stderr: -498> 2024-03-04T18:04:54.788+0000 7fbcfe233640 10 monclient(hunting): handle_auth_done global_id 6738 payload 274 2024-03-04T18:04:55.184 INFO:tasks.workunit.client.0.smithi137.stderr: -497> 2024-03-04T18:04:54.788+0000 7fbcfe233640 10 monclient: _finish_hunting 0 2024-03-04T18:04:55.184 INFO:tasks.workunit.client.0.smithi137.stderr: -496> 2024-03-04T18:04:54.788+0000 7fbcfe233640 1 monclient: found mon.noname-a 2024-03-04T18:04:55.184 INFO:tasks.workunit.client.0.smithi137.stderr: -495> 2024-03-04T18:04:54.788+0000 7fbcfe233640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.184 INFO:tasks.workunit.client.0.smithi137.stderr: -494> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.184 INFO:tasks.workunit.client.0.smithi137.stderr: -493> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:55.184 INFO:tasks.workunit.client.0.smithi137.stderr: -492> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: dump: 2024-03-04T18:04:55.184 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.184 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.184 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.184 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.184 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr: -491> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: _finish_auth 0 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr: -490> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: _check_auth_tickets 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr: -489> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 monclient: authenticate success, global_id 6738 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr: -488> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr: -487> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr: -486> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr: -485> 2024-03-04T18:04:54.788+0000 7fbbece09640 10 monclient: dump: 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.185 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr: -484> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr: -483> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 4 set_mon_vals no callback set 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr: -482> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 10 monclient: shutdown 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr: -481> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 asok(0x557c676ad130) unregister_commands rotate-key 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr: -480> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 10 monclient: build_initial_monmap 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr: -479> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 1 build_initial for_mkfs: 0 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr: -478> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 10 monclient: monmap: 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:55.186 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr: -477> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding auth protocol: cephx 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr: -476> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding auth protocol: cephx 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr: -475> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding auth protocol: cephx 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr: -474> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: secure 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr: -473> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: crc 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr: -472> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: secure 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr: -471> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: crc 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr: -470> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: secure 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr: -469> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: crc 2024-03-04T18:04:55.187 INFO:tasks.workunit.client.0.smithi137.stderr: -468> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: crc 2024-03-04T18:04:55.188 INFO:tasks.workunit.client.0.smithi137.stderr: -467> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: secure 2024-03-04T18:04:55.188 INFO:tasks.workunit.client.0.smithi137.stderr: -466> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: crc 2024-03-04T18:04:55.188 INFO:tasks.workunit.client.0.smithi137.stderr: -465> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: secure 2024-03-04T18:04:55.188 INFO:tasks.workunit.client.0.smithi137.stderr: -464> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: crc 2024-03-04T18:04:55.188 INFO:tasks.workunit.client.0.smithi137.stderr: -463> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c678917f0) adding con mode: secure 2024-03-04T18:04:55.188 INFO:tasks.workunit.client.0.smithi137.stderr: -462> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.188 INFO:tasks.workunit.client.0.smithi137.stderr: -461> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 1 librados: starting msgr at 2024-03-04T18:04:55.188 INFO:tasks.workunit.client.0.smithi137.stderr: -460> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 1 librados: starting objecter 2024-03-04T18:04:55.188 INFO:tasks.workunit.client.0.smithi137.stderr: -459> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command objecter_requests hook 0x557c6767df10 2024-03-04T18:04:55.188 INFO:tasks.workunit.client.0.smithi137.stderr: -458> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 1 librados: setting wanted keys 2024-03-04T18:04:55.188 INFO:tasks.workunit.client.0.smithi137.stderr: -457> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 1 librados: calling monclient init 2024-03-04T18:04:55.188 INFO:tasks.workunit.client.0.smithi137.stderr: -456> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 10 monclient: init 2024-03-04T18:04:55.188 INFO:tasks.workunit.client.0.smithi137.stderr: -455> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding auth protocol: cephx 2024-03-04T18:04:55.188 INFO:tasks.workunit.client.0.smithi137.stderr: -454> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding auth protocol: cephx 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -453> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding auth protocol: cephx 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -452> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: secure 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -451> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: crc 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -450> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: secure 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -449> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: crc 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -448> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: secure 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -447> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: crc 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -446> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: crc 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -445> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: secure 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -444> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: crc 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -443> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: secure 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -442> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: crc 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -441> 2024-03-04T18:04:54.788+0000 7fbd00f1c980 5 AuthRegistry(0x557c6788f990) adding con mode: secure 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -440> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.189 INFO:tasks.workunit.client.0.smithi137.stderr: -439> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -438> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command rotate-key hook 0x557c6788fad8 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -437> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -436> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient: _add_conns ranks=[0,2,1] 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -435> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-a con 0x557c678961f0 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -434> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-c con 0x557c678be6c0 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -433> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-b con 0x7fbce8003ac0 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -432> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -431> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -430> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -429> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -428> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient(hunting): authenticate will time out at 2527.912450s 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -427> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): get_auth_request con 0x557c678961f0 auth_method 0 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -426> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -425> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -424> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -423> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient(hunting): get_auth_request con 0x557c678be6c0 auth_method 0 2024-03-04T18:04:55.190 INFO:tasks.workunit.client.0.smithi137.stderr: -422> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -421> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -420> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -419> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -418> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -417> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -416> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -415> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -414> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -413> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient(hunting): handle_auth_done global_id 6741 payload 824 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -412> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient: _finish_hunting 0 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -411> 2024-03-04T18:04:54.792+0000 7fbcfea34640 1 monclient: found mon.noname-a 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -410> 2024-03-04T18:04:54.792+0000 7fbcfea34640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -409> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -408> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr: -407> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: dump: 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.191 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr: -406> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: _finish_auth 0 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr: -405> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: _check_auth_tickets 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr: -404> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 5 monclient: authenticate success, global_id 6741 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr: -403> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient: _renew_subs 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr: -402> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr: -401> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:55.192 INFO:tasks.workunit.client.0.smithi137.stderr: -400> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr: -399> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr: -398> 2024-03-04T18:04:54.792+0000 7fbbeb606640 10 monclient: dump: 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr: -397> 2024-03-04T18:04:54.792+0000 7fbbece09640 4 set_mon_vals no callback set 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr: -396> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient: _renew_subs 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr: -395> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.193 INFO:tasks.workunit.client.0.smithi137.stderr: -394> 2024-03-04T18:04:54.792+0000 7fbd00f1c980 1 librados: init done 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr: -393> 2024-03-04T18:04:54.792+0000 7fbbeb606640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr: -392> 2024-03-04T18:04:54.792+0000 7fbbeb606640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr: -391> 2024-03-04T18:04:54.792+0000 7fbbeb606640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr: -390> 2024-03-04T18:04:54.792+0000 7fbcfe233640 10 monclient: get_auth_request con 0x7fbbe4065970 auth_method 0 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr: -389> 2024-03-04T18:04:54.792+0000 7fbcff235640 10 monclient: get_auth_request con 0x557c678d82d0 auth_method 0 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr: -388> 2024-03-04T18:04:54.796+0000 7fbcfea34640 10 monclient: get_auth_request con 0x557c678db110 auth_method 0 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr: -387> 2024-03-04T18:04:54.796+0000 7fbcfe233640 10 monclient: get_auth_request con 0x557c678df300 auth_method 0 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr: -386> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 10 monclient: build_initial_monmap 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr: -385> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 1 build_initial for_mkfs: 0 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr: -384> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 10 monclient: monmap: 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:55.194 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.195 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:55.195 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:55.195 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:55.195 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.195 INFO:tasks.workunit.client.0.smithi137.stderr: -383> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding auth protocol: cephx 2024-03-04T18:04:55.195 INFO:tasks.workunit.client.0.smithi137.stderr: -382> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding auth protocol: cephx 2024-03-04T18:04:55.195 INFO:tasks.workunit.client.0.smithi137.stderr: -381> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding auth protocol: cephx 2024-03-04T18:04:55.195 INFO:tasks.workunit.client.0.smithi137.stderr: -380> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: secure 2024-03-04T18:04:55.195 INFO:tasks.workunit.client.0.smithi137.stderr: -379> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: crc 2024-03-04T18:04:55.195 INFO:tasks.workunit.client.0.smithi137.stderr: -378> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: secure 2024-03-04T18:04:55.195 INFO:tasks.workunit.client.0.smithi137.stderr: -377> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: crc 2024-03-04T18:04:55.195 INFO:tasks.workunit.client.0.smithi137.stderr: -376> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: secure 2024-03-04T18:04:55.195 INFO:tasks.workunit.client.0.smithi137.stderr: -375> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: crc 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -374> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: crc 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -373> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: secure 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -372> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: crc 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -371> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: secure 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -370> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: crc 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -369> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e3d00) adding con mode: secure 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -368> 2024-03-04T18:04:54.796+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -367> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command objecter_requests cmddesc objecter_requests hook 0x557c678e93f0 EEXIST 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -366> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: init 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -365> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding auth protocol: cephx 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -364> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding auth protocol: cephx 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -363> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding auth protocol: cephx 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -362> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: secure 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -361> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: crc 2024-03-04T18:04:55.196 INFO:tasks.workunit.client.0.smithi137.stderr: -360> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: secure 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -359> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: crc 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -358> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: secure 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -357> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: crc 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -356> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: crc 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -355> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: secure 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -354> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: crc 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -353> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: secure 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -352> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: crc 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -351> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c678e2518) adding con mode: secure 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -350> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -349> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -348> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command rotate-key cmddesc rotate-key hook 0x557c678e2660 EEXIST 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -347> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -346> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: _add_conns ranks=[0,1,2] 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -345> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-a con 0x557c6790e050 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:55.197 INFO:tasks.workunit.client.0.smithi137.stderr: -344> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-b con 0x557c6790e430 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -343> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-c con 0x557c679095f0 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -342> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -341> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -340> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -339> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -338> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient(hunting): authenticate will time out at 2527.920678s 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -337> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient(hunting): get_auth_request con 0x557c6790e050 auth_method 0 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -336> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -335> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -334> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -333> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): get_auth_request con 0x557c679095f0 auth_method 0 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -332> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -331> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -330> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -329> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.198 INFO:tasks.workunit.client.0.smithi137.stderr: -328> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr: -327> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr: -326> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr: -325> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr: -324> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr: -323> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient(hunting): handle_auth_done global_id 6386 payload 824 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr: -322> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient: _finish_hunting 0 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr: -321> 2024-03-04T18:04:54.800+0000 7fbcff235640 1 monclient: found mon.noname-c 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr: -320> 2024-03-04T18:04:54.800+0000 7fbcff235640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.137:3301/0 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr: -319> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr: -318> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr: -317> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: dump: 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.199 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr: -316> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: _finish_auth 0 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr: -315> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: _check_auth_tickets 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr: -314> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr: -313> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr: -312> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr: -311> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 10 monclient: dump: 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.200 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr: -310> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 monclient: authenticate success, global_id 6386 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr: -309> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: _renew_subs 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr: -308> 2024-03-04T18:04:54.800+0000 7fbcffa36640 4 set_mon_vals no callback set 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr: -307> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.137:3301/0 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr: -306> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: _renew_subs 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr: -305> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.137:3301/0 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr: -304> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr: -303> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr: -302> 2024-03-04T18:04:54.800+0000 7fbbe8e01640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr: -301> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: get_monmap_and_config 2024-03-04T18:04:55.201 INFO:tasks.workunit.client.0.smithi137.stderr: -300> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: build_initial_monmap 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr: -299> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 1 build_initial for_mkfs: 0 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr: -298> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 10 monclient: monmap: 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr: -297> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding auth protocol: cephx 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr: -296> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding auth protocol: cephx 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr: -295> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding auth protocol: cephx 2024-03-04T18:04:55.202 INFO:tasks.workunit.client.0.smithi137.stderr: -294> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -293> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -292> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -291> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -290> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -289> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -288> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -287> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -286> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -285> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -284> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -283> 2024-03-04T18:04:54.800+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -282> 2024-03-04T18:04:54.800+0000 7fbcfea34640 10 monclient: get_auth_request con 0x7fbbd8065880 auth_method 0 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -281> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -280> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: init 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -279> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding auth protocol: cephx 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -278> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding auth protocol: cephx 2024-03-04T18:04:55.203 INFO:tasks.workunit.client.0.smithi137.stderr: -277> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding auth protocol: cephx 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -276> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: secure 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -275> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: crc 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -274> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: secure 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -273> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: crc 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -272> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: secure 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -271> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: crc 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -270> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: crc 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -269> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: secure 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -268> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: crc 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -267> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: secure 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -266> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: crc 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -265> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec3269bf0) adding con mode: secure 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -264> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -263> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -262> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command rotate-key cmddesc rotate-key hook 0x7ffec3269d38 EEXIST 2024-03-04T18:04:55.204 INFO:tasks.workunit.client.0.smithi137.stderr: -261> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -260> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: _add_conns ranks=[2,1,0] 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -259> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-c con 0x557c679523b0 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -258> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-b con 0x557c6795ca70 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -257> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-a con 0x557c6792d0d0 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -256> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -255> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -254> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -253> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -252> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): authenticate will time out at 2527.924796s 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -251> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): get_auth_request con 0x557c6792d0d0 auth_method 0 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -250> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -249> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -248> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -247> 2024-03-04T18:04:54.804+0000 7fbcfea34640 10 monclient(hunting): get_auth_request con 0x557c679523b0 auth_method 0 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -246> 2024-03-04T18:04:54.804+0000 7fbcfea34640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.205 INFO:tasks.workunit.client.0.smithi137.stderr: -245> 2024-03-04T18:04:54.804+0000 7fbcfea34640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr: -244> 2024-03-04T18:04:54.804+0000 7fbcfea34640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr: -243> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr: -242> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr: -241> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr: -240> 2024-03-04T18:04:54.804+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr: -239> 2024-03-04T18:04:54.804+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr: -238> 2024-03-04T18:04:54.804+0000 7fbcfea34640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr: -237> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): handle_auth_done global_id 6747 payload 274 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr: -236> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient: _finish_hunting 0 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr: -235> 2024-03-04T18:04:54.804+0000 7fbcff235640 1 monclient: found mon.noname-a 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr: -234> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr: -233> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr: -232> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr: -231> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: dump: 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.206 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr: -230> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: _finish_auth 0 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr: -229> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: _check_auth_tickets 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr: -228> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr: -227> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr: -226> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr: -225> 2024-03-04T18:04:54.804+0000 7fbbe27fc640 10 monclient: dump: 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.207 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.208 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.208 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.208 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.208 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.208 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.208 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.208 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.208 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.208 INFO:tasks.workunit.client.0.smithi137.stderr: -224> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 monclient: authenticate success, global_id 6747 2024-03-04T18:04:55.208 INFO:tasks.workunit.client.0.smithi137.stderr: -223> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: get_monmap_and_config success 2024-03-04T18:04:55.208 INFO:tasks.workunit.client.0.smithi137.stderr: -222> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 4 set_mon_vals no callback set 2024-03-04T18:04:55.208 INFO:tasks.workunit.client.0.smithi137.stderr: -221> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: shutdown 2024-03-04T18:04:55.208 INFO:tasks.workunit.client.0.smithi137.stderr: -220> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: build_initial_monmap 2024-03-04T18:04:55.208 INFO:tasks.workunit.client.0.smithi137.stderr: -219> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 1 build_initial for_mkfs: 0 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr: -218> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: monmap: 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr: -217> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding auth protocol: cephx 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr: -216> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding auth protocol: cephx 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr: -215> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding auth protocol: cephx 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr: -214> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr: -213> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.209 INFO:tasks.workunit.client.0.smithi137.stderr: -212> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -211> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -210> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -209> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -208> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -207> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -206> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -205> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -204> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: crc 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -203> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792bba0) adding con mode: secure 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -202> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -201> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 1 librados: starting msgr at 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -200> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 1 librados: starting objecter 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -199> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command objecter_requests cmddesc objecter_requests hook 0x557c6795ca50 EEXIST 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -198> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 1 librados: setting wanted keys 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -197> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 1 librados: calling monclient init 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -196> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: init 2024-03-04T18:04:55.210 INFO:tasks.workunit.client.0.smithi137.stderr: -195> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding auth protocol: cephx 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -194> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding auth protocol: cephx 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -193> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding auth protocol: cephx 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -192> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: secure 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -191> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: crc 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -190> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: secure 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -189> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: crc 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -188> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: secure 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -187> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: crc 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -186> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: crc 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -185> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: secure 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -184> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: crc 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -183> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: secure 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -182> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: crc 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -181> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 AuthRegistry(0x557c6792ad70) adding con mode: secure 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -180> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.211 INFO:tasks.workunit.client.0.smithi137.stderr: -179> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.212 INFO:tasks.workunit.client.0.smithi137.stderr: -178> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command rotate-key cmddesc rotate-key hook 0x557c6792aeb8 EEXIST 2024-03-04T18:04:55.212 INFO:tasks.workunit.client.0.smithi137.stderr: -177> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: _reopen_session rank -1 2024-03-04T18:04:55.212 INFO:tasks.workunit.client.0.smithi137.stderr: -176> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient: _add_conns ranks=[1,0,2] 2024-03-04T18:04:55.212 INFO:tasks.workunit.client.0.smithi137.stderr: -175> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-b con 0x557c6792d0d0 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:04:55.212 INFO:tasks.workunit.client.0.smithi137.stderr: -174> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-a con 0x557c679523b0 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:04:55.212 INFO:tasks.workunit.client.0.smithi137.stderr: -173> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): picked mon.noname-c con 0x557c6795ca70 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:04:55.212 INFO:tasks.workunit.client.0.smithi137.stderr: -172> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.212 INFO:tasks.workunit.client.0.smithi137.stderr: -171> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.212 INFO:tasks.workunit.client.0.smithi137.stderr: -170> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): start opening mon connection 2024-03-04T18:04:55.212 INFO:tasks.workunit.client.0.smithi137.stderr: -169> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): _renew_subs 2024-03-04T18:04:55.212 INFO:tasks.workunit.client.0.smithi137.stderr: -168> 2024-03-04T18:04:54.804+0000 7fbd00f1c980 10 monclient(hunting): authenticate will time out at 2527.927381s 2024-03-04T18:04:55.212 INFO:tasks.workunit.client.0.smithi137.stderr: -167> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): get_auth_request con 0x557c679523b0 auth_method 0 2024-03-04T18:04:55.212 INFO:tasks.workunit.client.0.smithi137.stderr: -166> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.212 INFO:tasks.workunit.client.0.smithi137.stderr: -165> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -164> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -163> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): get_auth_request con 0x557c6795ca70 auth_method 0 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -162> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -161> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -160> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -159> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -158> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -157> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -156> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -155> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -154> 2024-03-04T18:04:54.804+0000 7fbcff235640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -153> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient(hunting): handle_auth_done global_id 6750 payload 824 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -152> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient: _finish_hunting 0 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -151> 2024-03-04T18:04:54.804+0000 7fbcfe233640 1 monclient: found mon.noname-a 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -150> 2024-03-04T18:04:54.804+0000 7fbcfe233640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.213 INFO:tasks.workunit.client.0.smithi137.stderr: -149> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr: -148> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr: -147> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: dump: 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr: -146> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: _finish_auth 0 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr: -145> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: _check_auth_tickets 2024-03-04T18:04:55.214 INFO:tasks.workunit.client.0.smithi137.stderr: -144> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr: -143> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr: -142> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr: -141> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 10 monclient: dump: 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr: -140> 2024-03-04T18:04:54.808+0000 7fbbe27fc640 4 set_mon_vals no callback set 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr: -139> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 5 monclient: authenticate success, global_id 6750 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr: -138> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 10 monclient: _renew_subs 2024-03-04T18:04:55.215 INFO:tasks.workunit.client.0.smithi137.stderr: -137> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -136> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 10 monclient: _renew_subs 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -135> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -134> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 1 librados: init done 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -133> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command cr dump hook 0x557c6795db58 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -132> 2024-03-04T18:04:54.808+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -131> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -130> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -129> 2024-03-04T18:04:54.808+0000 7fbbe0ff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -128> 2024-03-04T18:04:54.808+0000 7fbcfea34640 10 monclient: get_auth_request con 0x7fbbc8065a30 auth_method 0 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -127> 2024-03-04T18:04:54.808+0000 7fbcff235640 10 monclient: get_auth_request con 0x557c67979e60 auth_method 0 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -126> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -125> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 realm 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -124> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -123> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -122> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:55.216 INFO:tasks.workunit.client.0.smithi137.stderr: -121> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -120> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -119> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -118> 2024-03-04T18:04:54.812+0000 7fbcfea34640 10 monclient: get_auth_request con 0x557c6797e0d0 auth_method 0 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -117> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -116> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -115> 2024-03-04T18:04:54.812+0000 7fbcfe233640 10 monclient: get_auth_request con 0x557c67981910 auth_method 0 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -114> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -113> 2024-03-04T18:04:54.812+0000 7fbd00f1c980 20 searching for the correct realm 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -112> 2024-03-04T18:04:54.812+0000 7fbcff235640 10 monclient: get_auth_request con 0x557c67986610 auth_method 0 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -111> 2024-03-04T18:04:54.816+0000 7fbbe0ff9640 1 client.6750.objecter ms_handle_reset 0x557c67986610 session 0x557c67985f90 osd.4 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -110> 2024-03-04T18:04:54.816+0000 7fbbe0ff9640 10 monclient: _renew_subs 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -109> 2024-03-04T18:04:54.816+0000 7fbbe0ff9640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -108> 2024-03-04T18:04:54.816+0000 7fbcff235640 10 monclient: get_auth_request con 0x7fbbc80754d0 auth_method 0 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -107> 2024-03-04T18:04:54.816+0000 7fbcfea34640 10 monclient: get_auth_request con 0x7fbcf8108070 auth_method 0 2024-03-04T18:04:55.217 INFO:tasks.workunit.client.0.smithi137.stderr: -106> 2024-03-04T18:04:54.816+0000 7fbcfea34640 0 -- 172.21.15.137:0/1542649010 >> [v2:172.21.15.137:6808/3491386063,v1:172.21.15.137:6810/3491386063] conn(0x7fbcf8108070 msgr2=0x7fbcf80f1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -105> 2024-03-04T18:04:55.020+0000 7fbcfea34640 10 monclient: get_auth_request con 0x7fbcf8108070 auth_method 0 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -104> 2024-03-04T18:04:55.024+0000 7fbcfe233640 10 monclient: get_auth_request con 0x7fbcf0010040 auth_method 0 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -103> 2024-03-04T18:04:55.024+0000 7fbcff235640 10 monclient: get_auth_request con 0x557c67744fd0 auth_method 0 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -102> 2024-03-04T18:04:55.028+0000 7fbcff235640 0 -- 172.21.15.137:0/1542649010 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x557c67979e60 msgr2=0x557c6797c2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -101> 2024-03-04T18:04:55.028+0000 7fbbe0ff9640 1 client.6750.objecter ms_handle_reset 0x557c67979e60 session 0x557c679797e0 osd.6 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -100> 2024-03-04T18:04:55.028+0000 7fbcff235640 10 monclient: get_auth_request con 0x557c67986610 auth_method 0 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -99> 2024-03-04T18:04:55.032+0000 7fbcfea34640 10 monclient: get_auth_request con 0x7fbcf0053470 auth_method 0 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -98> 2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -97> 2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -96> 2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -95> 2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -94> 2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -93> 2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -92> 2024-03-04T18:04:55.044+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.218 INFO:tasks.workunit.client.0.smithi137.stderr: -91> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -90> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -89> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -88> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -87> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -86> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 zone default found 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -85> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 4 Realm: () 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -84> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -83> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -82> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -81> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 zonegroup default 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -80> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -79> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -78> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -77> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -76> 2024-03-04T18:04:55.048+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.219 INFO:tasks.workunit.client.0.smithi137.stderr: -75> 2024-03-04T18:04:55.052+0000 7fbd00f1c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.220 INFO:tasks.workunit.client.0.smithi137.stderr: -74> 2024-03-04T18:04:55.052+0000 7fbd00f1c980 20 started sync module instance, tier type = 2024-03-04T18:04:55.220 INFO:tasks.workunit.client.0.smithi137.stderr: -73> 2024-03-04T18:04:55.052+0000 7fbd00f1c980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:55.220 INFO:tasks.workunit.client.0.smithi137.stderr: -72> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=0 2024-03-04T18:04:55.220 INFO:tasks.workunit.client.0.smithi137.stderr: -71> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=1 2024-03-04T18:04:55.220 INFO:tasks.workunit.client.0.smithi137.stderr: -70> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=2 2024-03-04T18:04:55.220 INFO:tasks.workunit.client.0.smithi137.stderr: -69> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=3 2024-03-04T18:04:55.220 INFO:tasks.workunit.client.0.smithi137.stderr: -68> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=4 2024-03-04T18:04:55.220 INFO:tasks.workunit.client.0.smithi137.stderr: -67> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=5 2024-03-04T18:04:55.220 INFO:tasks.workunit.client.0.smithi137.stderr: -66> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=6 2024-03-04T18:04:55.220 INFO:tasks.workunit.client.0.smithi137.stderr: -65> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 20 add_watcher() i=7 2024-03-04T18:04:55.220 INFO:tasks.workunit.client.0.smithi137.stderr: -64> 2024-03-04T18:04:55.076+0000 7fbd00f1c980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:55.220 INFO:tasks.workunit.client.0.smithi137.stderr: -63> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding auth protocol: cephx 2024-03-04T18:04:55.220 INFO:tasks.workunit.client.0.smithi137.stderr: -62> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding auth protocol: cephx 2024-03-04T18:04:55.220 INFO:tasks.workunit.client.0.smithi137.stderr: -61> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding auth protocol: cephx 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -60> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: secure 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -59> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: crc 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -58> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: secure 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -57> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: crc 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -56> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: secure 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -55> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: crc 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -54> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: crc 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -53> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: secure 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -52> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: crc 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -51> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: secure 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -50> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: crc 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -49> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 AuthRegistry(0x7ffec326a080) adding con mode: secure 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -48> 2024-03-04T18:04:55.084+0000 7fbbb57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -47> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -46> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:55.221 INFO:tasks.workunit.client.0.smithi137.stderr: -45> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -44> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command cache list hook 0x557c6792d4d0 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -43> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command cache inspect hook 0x557c6792d4d0 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -42> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command cache erase hook 0x557c6792d4d0 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -41> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command cache zap hook 0x557c6792d4d0 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -40> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 note: GC not initialized 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -39> 2024-03-04T18:04:55.084+0000 7fbb2ffff640 20 reqs_thread_entry: start 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -38> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command sync trace show hook 0x7fbcf8002170 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -37> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command sync trace history hook 0x7fbcf8002170 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -36> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command sync trace active hook 0x7fbcf8002170 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -35> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 5 asok(0x557c676ad130) register_command sync trace active_short hook 0x7fbcf8002170 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -34> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -33> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 Filter name: none 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -32> 2024-03-04T18:04:55.084+0000 7fbb2dffb640 20 reqs_thread_entry: start 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -31> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -30> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.222 INFO:tasks.workunit.client.0.smithi137.stderr: -29> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -28> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -27> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -26> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 : miss 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -25> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -24> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -23> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 info.flags=0x17 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -22> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 to cache LRU end 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -21> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -20> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -19> 2024-03-04T18:04:55.084+0000 7fbd00f1c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -18> 2024-03-04T18:04:55.092+0000 7fbd00f1c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 info.flags=0x17 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -17> 2024-03-04T18:04:55.092+0000 7fbd00f1c980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 to cache LRU end 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -16> 2024-03-04T18:04:55.092+0000 7fbd00f1c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -15> 2024-03-04T18:04:55.092+0000 7fbd00f1c980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27, ofs0, ns] 2024-03-04T18:04:55.223 INFO:tasks.workunit.client.0.smithi137.stderr: -14> 2024-03-04T18:04:55.096+0000 7fbbe1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627822 cookie 93992802367728 notifier 6750 bl.length()=965 2024-03-04T18:04:55.224 INFO:tasks.workunit.client.0.smithi137.stderr: -13> 2024-03-04T18:04:55.096+0000 7fbbe1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 info.flags=0x17 2024-03-04T18:04:55.224 INFO:tasks.workunit.client.0.smithi137.stderr: -12> 2024-03-04T18:04:55.096+0000 7fbbe1ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 to cache LRU end 2024-03-04T18:04:55.224 INFO:tasks.workunit.client.0.smithi137.stderr: -11> 2024-03-04T18:04:55.096+0000 7fbbe1ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:04:55.224 INFO:tasks.workunit.client.0.smithi137.stderr: -10> 2024-03-04T18:04:55.096+0000 7fbd00f1c980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): orig_sources= new_sources= 2024-03-04T18:04:55.224 INFO:tasks.workunit.client.0.smithi137.stderr: -9> 2024-03-04T18:04:55.096+0000 7fbd00f1c980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): potential sources added=[] removed=[] 2024-03-04T18:04:55.224 INFO:tasks.workunit.client.0.smithi137.stderr: -8> 2024-03-04T18:04:55.096+0000 7fbd00f1c980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): orig_dests= new_dests= 2024-03-04T18:04:55.224 INFO:tasks.workunit.client.0.smithi137.stderr: -7> 2024-03-04T18:04:55.096+0000 7fbd00f1c980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): potential dests added=[] removed=[] 2024-03-04T18:04:55.224 INFO:tasks.workunit.client.0.smithi137.stderr: -6> 2024-03-04T18:04:55.100+0000 7fbd00f1c980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.0 2024-03-04T18:04:55.224 INFO:tasks.workunit.client.0.smithi137.stderr: -5> 2024-03-04T18:04:55.100+0000 7fbd00f1c980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.0 2024-03-04T18:04:55.224 INFO:tasks.workunit.client.0.smithi137.stderr: -4> 2024-03-04T18:04:55.100+0000 7fbd00f1c980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.1 2024-03-04T18:04:55.224 INFO:tasks.workunit.client.0.smithi137.stderr: -3> 2024-03-04T18:04:55.100+0000 7fbd00f1c980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.1 2024-03-04T18:04:55.224 INFO:tasks.workunit.client.0.smithi137.stderr: -2> 2024-03-04T18:04:55.100+0000 7fbd00f1c980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.0 2024-03-04T18:04:55.224 INFO:tasks.workunit.client.0.smithi137.stderr: -1> 2024-03-04T18:04:55.104+0000 7fbd00f1c980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fbd00f1c980 time 2024-03-04T18:04:55.107872+0000 2024-03-04T18:04:55.224 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:04:55.225 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.225 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:55.225 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fbd047f1d7b] 2024-03-04T18:04:55.225 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fbd047f1f2f] 2024-03-04T18:04:55.225 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557c64d7c281] 2024-03-04T18:04:55.225 INFO:tasks.workunit.client.0.smithi137.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x557c64d88fbf] 2024-03-04T18:04:55.225 INFO:tasks.workunit.client.0.smithi137.stderr: 5: main() 2024-03-04T18:04:55.225 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd0365cd90] 2024-03-04T18:04:55.225 INFO:tasks.workunit.client.0.smithi137.stderr: 7: __libc_start_main() 2024-03-04T18:04:55.225 INFO:tasks.workunit.client.0.smithi137.stderr: 8: _start() 2024-03-04T18:04:55.225 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.225 INFO:tasks.workunit.client.0.smithi137.stderr: 0> 2024-03-04T18:04:55.104+0000 7fbd00f1c980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:04:55.225 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7fbd00f1c980 thread_name:radosgw-admin 2024-03-04T18:04:55.225 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fbd03675520] 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fbd047f1dd5] 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7fbd047f1f2f] 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557c64d7c281] 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x56f) [0x557c64d88fbf] 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: 9: main() 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbd0365cd90] 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: 11: __libc_start_main() 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: 12: _start() 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.226 INFO:tasks.workunit.client.0.smithi137.stderr:--- logging levels --- 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 none 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 lockdep 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 context 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 crush 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_balancer 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_locker 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log_expire 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_migrator 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 buffer 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 timer 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 filer 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 striper 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 objecter 2024-03-04T18:04:55.227 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rados 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_mirror 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_replay 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_pwl 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 journaler 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objectcacher 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 client 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 osd 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 optracker 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objclass 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 filestore 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 journal 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 ms 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mon 2024-03-04T18:04:55.228 INFO:tasks.workunit.client.0.smithi137.stderr: 0/10 monc 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 paxos 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 tp 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 auth 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 crypto 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 finisher 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 reserver 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 heartbeatmap 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 perfcounter 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 20/20 rgw 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_sync 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_datacache 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_access 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_flight 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 javaclient 2024-03-04T18:04:55.229 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 asok 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 throttle 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 refs 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 compressor 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluestore 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluefs 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 bdev 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 kstore 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 4/ 5 rocksdb 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 fuse 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 2/ 5 mgr 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mgrc 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 dpdk 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 eventtrace 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 prioritycache 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 test 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:04:55.230 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephsqlite 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_onode 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_odata 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_omap 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_tm 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_t 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_epm 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_lba 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cache 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_journal 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_device 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_backref 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 alienstore 2024-03-04T18:04:55.231 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mclock 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cyanstore 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 ceph_exporter 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 memstore 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 trace 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: -2/-2 (syslog threshold) 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: 99/99 (stderr threshold) 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbb2dffb640 / http_manager 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbb2ffff640 / http_manager 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbbb57e2640 / rgw_dt_lg_renew 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbbe0ff9640 / ms_dispatch 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbbe1ffb640 / io_context_pool 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbbe27fc640 / io_context_pool 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbbe8e01640 / ms_dispatch 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbbeb606640 / ms_dispatch 2024-03-04T18:04:55.232 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbbece09640 / io_context_pool 2024-03-04T18:04:55.233 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbcfe233640 / msgr-worker-2 2024-03-04T18:04:55.233 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbcfea34640 / msgr-worker-1 2024-03-04T18:04:55.233 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbcff235640 / msgr-worker-0 2024-03-04T18:04:55.233 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbcffa36640 / io_context_pool 2024-03-04T18:04:55.233 INFO:tasks.workunit.client.0.smithi137.stderr: 7fbd00f1c980 / radosgw-admin 2024-03-04T18:04:55.233 INFO:tasks.workunit.client.0.smithi137.stderr: max_recent 500 2024-03-04T18:04:55.233 INFO:tasks.workunit.client.0.smithi137.stderr: max_new 1000 2024-03-04T18:04:55.233 INFO:tasks.workunit.client.0.smithi137.stderr: log_file /var/lib/ceph/crash/2024-03-04T18:04:55.111422Z_e8855bfa-f4a7-4736-acd5-fe0d6d90d840/log 2024-03-04T18:04:55.233 INFO:tasks.workunit.client.0.smithi137.stderr:--- end dump of recent events --- 2024-03-04T18:04:55.233 INFO:tasks.workunit.client.0.smithi137.stderr:Aborted 2024-03-04T18:04:55.233 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-03-04T18:04:55.235 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:04:55.236 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:04:55.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.192+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.192+0000 7f4445e0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.192+0000 7f4445e0a980 20 realm 2024-03-04T18:04:55.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.192+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.196+0000 7f4445e0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.196+0000 7f4445e0a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:04:55.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.196+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.196+0000 7f4445e0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.196+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.196+0000 7f4445e0a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:55.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.196+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.236 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.196+0000 7f4445e0a980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.196+0000 7f4445e0a980 20 searching for the correct realm 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.200+0000 7f4444123640 0 -- 172.21.15.137:0/2943736249 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x55589f1855b0 msgr2=0x55589f187a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.204+0000 7f4443922640 0 -- 172.21.15.137:0/2943736249 >> [v2:172.21.15.137:6824/1039931256,v1:172.21.15.137:6825/1039931256] conn(0x55589f1896f0 msgr2=0x55589f18bbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:04:55.237 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 zone default found 2024-03-04T18:04:55.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 4 Realm: () 2024-03-04T18:04:55.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:04:55.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:04:55.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:04:55.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 zonegroup default 2024-03-04T18:04:55.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.216+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.220+0000 7f4445e0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.220+0000 7f4445e0a980 20 started sync module instance, tier type = 2024-03-04T18:04:55.238 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.220+0000 7f4445e0a980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:04:55.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.236+0000 7f4445e0a980 20 add_watcher() i=0 2024-03-04T18:04:55.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.236+0000 7f4445e0a980 20 add_watcher() i=1 2024-03-04T18:04:55.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.236+0000 7f4445e0a980 20 add_watcher() i=2 2024-03-04T18:04:55.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.236+0000 7f4445e0a980 20 add_watcher() i=3 2024-03-04T18:04:55.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.236+0000 7f4445e0a980 20 add_watcher() i=4 2024-03-04T18:04:55.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.236+0000 7f4445e0a980 20 add_watcher() i=5 2024-03-04T18:04:55.239 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.236+0000 7f4445e0a980 20 add_watcher() i=6 2024-03-04T18:04:55.241 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.240+0000 7f4445e0a980 20 add_watcher() i=7 2024-03-04T18:04:55.241 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.240+0000 7f4445e0a980 2 all 8 watchers are set, enabling cache 2024-03-04T18:04:55.245 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.244+0000 7f42f9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:55.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.244+0000 7f4445e0a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:04:55.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.244+0000 7f4445e0a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:04:55.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.244+0000 7f4445e0a980 5 note: GC not initialized 2024-03-04T18:04:55.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.244+0000 7f42f8fe1640 20 reqs_thread_entry: start 2024-03-04T18:04:55.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.244+0000 7f4445e0a980 20 init_complete bucket index max shards: 1 2024-03-04T18:04:55.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.244+0000 7f4445e0a980 20 Filter name: none 2024-03-04T18:04:55.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.244+0000 7f42727fc640 20 reqs_thread_entry: start 2024-03-04T18:04:55.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.244+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.246 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.244+0000 7f4445e0a980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:55.247 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.244+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.247 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.244+0000 7f4445e0a980 20 rados_obj.operate() r=0 bl.length=425 2024-03-04T18:04:55.247 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.244+0000 7f4445e0a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:55.247 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.244+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.248+0000 7f4445e0a980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:04:55.248 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.248+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.248+0000 7f4445e0a980 20 rados_obj.operate() r=0 bl.length=425 2024-03-04T18:04:55.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.248+0000 7f4445e0a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:04:55.249 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.248+0000 7f4445e0a980 20 rados->read ofs=0 len=0 2024-03-04T18:04:55.250 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.248+0000 7f4445e0a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:04:55.251 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.248+0000 7f42f9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:04:55.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.252+0000 7f4445e0a980 20 remove_watcher() i=0 2024-03-04T18:04:55.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.252+0000 7f4445e0a980 2 removed watcher, disabling cache 2024-03-04T18:04:55.257 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.256+0000 7f4445e0a980 20 remove_watcher() i=1 2024-03-04T18:04:55.259 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.256+0000 7f4445e0a980 20 remove_watcher() i=2 2024-03-04T18:04:55.261 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.260+0000 7f4445e0a980 20 remove_watcher() i=3 2024-03-04T18:04:55.263 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.260+0000 7f4445e0a980 20 remove_watcher() i=4 2024-03-04T18:04:55.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.264+0000 7f4445e0a980 20 remove_watcher() i=5 2024-03-04T18:04:55.267 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.264+0000 7f4445e0a980 20 remove_watcher() i=6 2024-03-04T18:04:55.270 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:04:55.268+0000 7f4445e0a980 20 remove_watcher() i=7 2024-03-04T18:04:55.284 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.284 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:04:55.284 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: abort-at-commit-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:04:55.284 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:04:55.285 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:04:55.285 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:25.508 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:25.509 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:25.594 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:05:25.630 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:25.630 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:25.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.672+0000 7fa78223e980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.675 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.672+0000 7fa78223e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.672+0000 7fa78223e980 20 realm 2024-03-04T18:05:25.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.672+0000 7fa78223e980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.676+0000 7fa78223e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.676+0000 7fa78223e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:25.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.676+0000 7fa78223e980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.676+0000 7fa78223e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.677 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.676+0000 7fa78223e980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.676+0000 7fa78223e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:25.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.676+0000 7fa78223e980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.676+0000 7fa78223e980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:25.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.676+0000 7fa78223e980 20 searching for the correct realm 2024-03-04T18:05:25.684 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.684+0000 7fa780557640 0 -- 172.21.15.137:0/1035258000 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fa778086290 msgr2=0x7fa7780700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:25.902 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.900+0000 7fa780557640 0 -- 172.21.15.137:0/1035258000 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55ccf491bca0 msgr2=0x55ccf491e160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:25.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.908+0000 7fa78223e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:25.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.908+0000 7fa78223e980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:05:25.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.908+0000 7fa78223e980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:25.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.908+0000 7fa78223e980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:25.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.908+0000 7fa78223e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:25.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.908+0000 7fa78223e980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:05:25.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.908+0000 7fa78223e980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:25.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:25.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 20 zone default found 2024-03-04T18:05:25.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 4 Realm: () 2024-03-04T18:05:25.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:05:25.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:05:25.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:25.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 20 zonegroup default 2024-03-04T18:05:25.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.912+0000 7fa78223e980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.916 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.916+0000 7fa78223e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:25.916 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.916+0000 7fa78223e980 20 started sync module instance, tier type = 2024-03-04T18:05:25.917 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.916+0000 7fa78223e980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:05:25.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.940+0000 7fa78223e980 20 add_watcher() i=0 2024-03-04T18:05:25.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.940+0000 7fa78223e980 20 add_watcher() i=1 2024-03-04T18:05:25.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.940+0000 7fa78223e980 20 add_watcher() i=2 2024-03-04T18:05:25.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.940+0000 7fa78223e980 20 add_watcher() i=3 2024-03-04T18:05:25.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.940+0000 7fa78223e980 20 add_watcher() i=4 2024-03-04T18:05:25.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.940+0000 7fa78223e980 20 add_watcher() i=5 2024-03-04T18:05:25.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.940+0000 7fa78223e980 20 add_watcher() i=6 2024-03-04T18:05:25.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.940+0000 7fa78223e980 20 add_watcher() i=7 2024-03-04T18:05:25.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.940+0000 7fa78223e980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:25.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.944+0000 7fa635fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:25.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.944+0000 7fa78223e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:25.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.944+0000 7fa78223e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:25.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.944+0000 7fa78223e980 5 note: GC not initialized 2024-03-04T18:05:25.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa634fe1640 20 reqs_thread_entry: start 2024-03-04T18:05:25.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 20 init_complete bucket index max shards: 1 2024-03-04T18:05:25.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 20 Filter name: none 2024-03-04T18:05:25.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa5ae7fc640 20 reqs_thread_entry: start 2024-03-04T18:05:25.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-03-04T18:05:25.949 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.950 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:05:25.950 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-03-04T18:05:25.950 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-03-04T18:05:25.950 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 : miss 2024-03-04T18:05:25.950 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 20 rados->read ofs=0 len=0 2024-03-04T18:05:25.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:05:25.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 info.flags=0x17 2024-03-04T18:05:25.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 to cache LRU end 2024-03-04T18:05:25.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:05:25.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:25.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.948+0000 7fa78223e980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 2024-03-04T18:05:25.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.956+0000 7fa78223e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 info.flags=0x17 2024-03-04T18:05:25.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.956+0000 7fa78223e980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 to cache LRU end 2024-03-04T18:05:25.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.956+0000 7fa78223e980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:25.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.956+0000 7fa78223e980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27, ofs0, ns] 2024-03-04T18:05:25.959 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.956+0000 7fa65effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627824 cookie 94338764909360 notifier 6786 bl.length()=965 2024-03-04T18:05:25.959 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.956+0000 7fa65effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 info.flags=0x17 2024-03-04T18:05:25.959 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.956+0000 7fa65effd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 to cache LRU end 2024-03-04T18:05:25.959 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.956+0000 7fa65effd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:25.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.960+0000 7fa78223e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): orig_sources= new_sources= 2024-03-04T18:05:25.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.960+0000 7fa78223e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): potential sources added=[] removed=[] 2024-03-04T18:05:25.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.960+0000 7fa78223e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): orig_dests= new_dests= 2024-03-04T18:05:25.961 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.960+0000 7fa78223e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): potential dests added=[] removed=[] 2024-03-04T18:05:25.963 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.960+0000 7fa78223e980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.0 2024-03-04T18:05:25.963 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.960+0000 7fa78223e980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.0 2024-03-04T18:05:25.963 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.960+0000 7fa78223e980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.1 2024-03-04T18:05:25.963 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.960+0000 7fa78223e980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.1.1 2024-03-04T18:05:25.963 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.960+0000 7fa78223e980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27.0 2024-03-04T18:05:25.967 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.964+0000 7fa78223e980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 : hit (requested=0x17, cached=0x17) 2024-03-04T18:05:25.967 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.964+0000 7fa78223e980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 2024-03-04T18:05:25.969 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.968+0000 7fa78223e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 info.flags=0x17 2024-03-04T18:05:25.969 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.968+0000 7fa78223e980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 to cache LRU end 2024-03-04T18:05:25.969 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.968+0000 7fa78223e980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:25.969 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.968+0000 7fa78223e980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27, ofs0, ns] 2024-03-04T18:05:25.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.968+0000 7fa65e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627825 cookie 94338764909360 notifier 6786 bl.length()=933 2024-03-04T18:05:25.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.968+0000 7fa65e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 info.flags=0x17 2024-03-04T18:05:25.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.968+0000 7fa65e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 to cache LRU end 2024-03-04T18:05:25.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.968+0000 7fa65e7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:25.972 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.972+0000 7fa78223e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): orig_sources= new_sources= 2024-03-04T18:05:25.972 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.972+0000 7fa78223e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): potential sources added=[] removed=[] 2024-03-04T18:05:25.972 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.972+0000 7fa78223e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): orig_dests= new_dests= 2024-03-04T18:05:25.973 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.972+0000 7fa78223e980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27]): potential dests added=[] removed=[] 2024-03-04T18:05:25.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.972+0000 7fa78223e980 1 execute INFO: reshard of bucket "abort-at-commit-target-layout" completed successfully 2024-03-04T18:05:25.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.976+0000 7fa635fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:25.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.980+0000 7fa78223e980 20 remove_watcher() i=0 2024-03-04T18:05:25.982 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.980+0000 7fa78223e980 2 removed watcher, disabling cache 2024-03-04T18:05:25.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.984+0000 7fa78223e980 20 remove_watcher() i=1 2024-03-04T18:05:25.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.984+0000 7fa78223e980 20 remove_watcher() i=2 2024-03-04T18:05:25.988 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.988+0000 7fa78223e980 20 remove_watcher() i=3 2024-03-04T18:05:25.990 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.988+0000 7fa78223e980 20 remove_watcher() i=4 2024-03-04T18:05:25.993 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.992+0000 7fa78223e980 20 remove_watcher() i=5 2024-03-04T18:05:25.996 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.992+0000 7fa78223e980 20 remove_watcher() i=6 2024-03-04T18:05:25.998 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:25.996+0000 7fa78223e980 20 remove_watcher() i=7 2024-03-04T18:05:26.014 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-03-04T18:05:26.044 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:26.044 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:26.087 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.084+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.088+0000 7fc7d1f4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.088+0000 7fc7d1f4b980 20 realm 2024-03-04T18:05:26.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.088+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.088+0000 7fc7d1f4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.088+0000 7fc7d1f4b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:26.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.088+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.088+0000 7fc7d1f4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.088+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.088+0000 7fc7d1f4b980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:26.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.088+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.091 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.088+0000 7fc7d1f4b980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:26.092 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.088+0000 7fc7d1f4b980 20 searching for the correct realm 2024-03-04T18:05:26.096 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.096+0000 7fc7d0264640 0 -- 172.21.15.137:0/3691706709 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55c08fe1a440 msgr2=0x55c08fe1c900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:26.102 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.100+0000 7fc7cf262640 0 -- 172.21.15.137:0/3691706709 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x55c08fe15780 msgr2=0x55c08fe17b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:26.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.108+0000 7fc7d1f4b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:26.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.108+0000 7fc7d1f4b980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:05:26.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.108+0000 7fc7d1f4b980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:26.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.108+0000 7fc7d1f4b980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:26.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.108+0000 7fc7d1f4b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:26.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.108+0000 7fc7d1f4b980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:05:26.111 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.108+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.113 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.108+0000 7fc7d1f4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.108+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.108+0000 7fc7d1f4b980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:26.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.108+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:26.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 20 zone default found 2024-03-04T18:05:26.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 4 Realm: () 2024-03-04T18:05:26.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:05:26.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:05:26.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:26.114 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 20 zonegroup default 2024-03-04T18:05:26.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.115 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 20 started sync module instance, tier type = 2024-03-04T18:05:26.116 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.112+0000 7fc7d1f4b980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:05:26.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.128+0000 7fc7d1f4b980 20 add_watcher() i=0 2024-03-04T18:05:26.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.128+0000 7fc7d1f4b980 20 add_watcher() i=1 2024-03-04T18:05:26.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.128+0000 7fc7d1f4b980 20 add_watcher() i=2 2024-03-04T18:05:26.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.128+0000 7fc7d1f4b980 20 add_watcher() i=3 2024-03-04T18:05:26.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.128+0000 7fc7d1f4b980 20 add_watcher() i=4 2024-03-04T18:05:26.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.128+0000 7fc7d1f4b980 20 add_watcher() i=5 2024-03-04T18:05:26.131 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.128+0000 7fc7d1f4b980 20 add_watcher() i=6 2024-03-04T18:05:26.133 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.132+0000 7fc7d1f4b980 20 add_watcher() i=7 2024-03-04T18:05:26.133 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.132+0000 7fc7d1f4b980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:26.137 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.136+0000 7fc685fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:26.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.136+0000 7fc7d1f4b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:26.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.136+0000 7fc7d1f4b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:26.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.136+0000 7fc7d1f4b980 5 note: GC not initialized 2024-03-04T18:05:26.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.136+0000 7fc684fe1640 20 reqs_thread_entry: start 2024-03-04T18:05:26.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.136+0000 7fc7d1f4b980 20 init_complete bucket index max shards: 1 2024-03-04T18:05:26.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.136+0000 7fc7d1f4b980 20 Filter name: none 2024-03-04T18:05:26.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.136+0000 7fc5fe7fc640 20 reqs_thread_entry: start 2024-03-04T18:05:26.138 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.136+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.136+0000 7fc7d1f4b980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:05:26.139 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.136+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.136+0000 7fc7d1f4b980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:05:26.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.136+0000 7fc7d1f4b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:26.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.136+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.140+0000 7fc7d1f4b980 20 rados_obj.operate() r=0 bl.length=221 2024-03-04T18:05:26.141 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.140+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.141 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.140+0000 7fc7d1f4b980 20 rados_obj.operate() r=0 bl.length=393 2024-03-04T18:05:26.141 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.140+0000 7fc7d1f4b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:26.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.140+0000 7fc7d1f4b980 20 rados->read ofs=0 len=0 2024-03-04T18:05:26.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.140+0000 7fc7d1f4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:26.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.140+0000 7fc685fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:26.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.144+0000 7fc7d1f4b980 20 remove_watcher() i=0 2024-03-04T18:05:26.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.144+0000 7fc7d1f4b980 2 removed watcher, disabling cache 2024-03-04T18:05:26.149 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.148+0000 7fc7d1f4b980 20 remove_watcher() i=1 2024-03-04T18:05:26.151 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.148+0000 7fc7d1f4b980 20 remove_watcher() i=2 2024-03-04T18:05:26.154 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.152+0000 7fc7d1f4b980 20 remove_watcher() i=3 2024-03-04T18:05:26.155 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.152+0000 7fc7d1f4b980 20 remove_watcher() i=4 2024-03-04T18:05:26.157 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.156+0000 7fc7d1f4b980 20 remove_watcher() i=5 2024-03-04T18:05:26.160 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.160+0000 7fc7d1f4b980 20 remove_watcher() i=6 2024-03-04T18:05:26.163 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:26.160+0000 7fc7d1f4b980 20 remove_watcher() i=7 2024-03-04T18:05:26.177 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:26.178 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:05:26.178 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: abort-at-commit-target-layout id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.27 num_objects: 19 size_kb: 1 num_shards: 2 2024-03-04T18:05:26.178 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:26.179 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:26.179 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:26.200 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:26.200 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:26.527 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:26.527 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:26.696 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: reshard bucket with EIO injected at do_reshard 2024-03-04T18:05:26.696 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:26.699 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:26.699 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:26.809 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:26.809 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:26.892 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:26.892 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:26.977 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.015 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:27.062 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.062 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:27.149 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.149 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:27.229 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.229 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:27.315 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.316 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:27.399 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.399 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:27.484 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.484 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:27.565 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.565 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:27.647 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.647 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:27.730 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.730 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:27.815 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.815 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:27.899 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.899 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:27.984 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:27.984 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:28.066 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.066 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:28.152 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.152 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:28.237 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.237 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:28.326 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.326 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:28.407 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.407 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:28.491 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:28.492 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:28.576 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-03-04T18:05:28.608 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:28.608 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:28.651 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.648+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.652+0000 7fa2e5ee5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:28.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.652+0000 7fa2e5ee5980 20 realm 2024-03-04T18:05:28.654 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.652+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.655 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.652+0000 7fa2e5ee5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:28.655 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.652+0000 7fa2e5ee5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:28.655 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.652+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.655 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.652+0000 7fa2e5ee5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:28.655 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.652+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.655 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.652+0000 7fa2e5ee5980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:28.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.652+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.656+0000 7fa2e5ee5980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:28.656 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.656+0000 7fa2e5ee5980 20 searching for the correct realm 2024-03-04T18:05:28.662 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.660+0000 7fa2e41fe640 0 -- 172.21.15.137:0/2488928706 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fa2dc0aa8e0 msgr2=0x7fa2dc0c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:28.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.876+0000 7fa2e41fe640 0 -- 172.21.15.137:0/2488928706 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x56380c113bd0 msgr2=0x56380c116090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:28.887 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.884+0000 7fa2e5ee5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:28.887 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.884+0000 7fa2e5ee5980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:05:28.887 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.884+0000 7fa2e5ee5980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:28.887 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.884+0000 7fa2e5ee5980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:28.887 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.884+0000 7fa2e5ee5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:28.887 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.884+0000 7fa2e5ee5980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:05:28.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.884+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.884+0000 7fa2e5ee5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:28.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.884+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:28.888 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:28.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 20 zone default found 2024-03-04T18:05:28.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 4 Realm: () 2024-03-04T18:05:28.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:05:28.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:05:28.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:28.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 20 zonegroup default 2024-03-04T18:05:28.889 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:28.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:28.890 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.891 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:28.891 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 20 started sync module instance, tier type = 2024-03-04T18:05:28.892 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.888+0000 7fa2e5ee5980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:05:28.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.912+0000 7fa2e5ee5980 20 add_watcher() i=0 2024-03-04T18:05:28.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.912+0000 7fa2e5ee5980 20 add_watcher() i=1 2024-03-04T18:05:28.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.912+0000 7fa2e5ee5980 20 add_watcher() i=2 2024-03-04T18:05:28.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.912+0000 7fa2e5ee5980 20 add_watcher() i=3 2024-03-04T18:05:28.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.912+0000 7fa2e5ee5980 20 add_watcher() i=4 2024-03-04T18:05:28.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.912+0000 7fa2e5ee5980 20 add_watcher() i=5 2024-03-04T18:05:28.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.912+0000 7fa2e5ee5980 20 add_watcher() i=6 2024-03-04T18:05:28.916 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.916+0000 7fa2e5ee5980 20 add_watcher() i=7 2024-03-04T18:05:28.916 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.916+0000 7fa2e5ee5980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:28.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.920+0000 7fa199fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:28.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.920+0000 7fa2e5ee5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:28.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.920+0000 7fa2e5ee5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:28.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.920+0000 7fa2e5ee5980 5 note: GC not initialized 2024-03-04T18:05:28.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.920+0000 7fa198fe1640 20 reqs_thread_entry: start 2024-03-04T18:05:28.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.920+0000 7fa2e5ee5980 20 init_complete bucket index max shards: 1 2024-03-04T18:05:28.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.920+0000 7fa2e5ee5980 20 Filter name: none 2024-03-04T18:05:28.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.920+0000 7fa1127fc640 20 reqs_thread_entry: start 2024-03-04T18:05:28.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.920+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.924 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.924+0000 7fa2e5ee5980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:28.924 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.924+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.925 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.924+0000 7fa2e5ee5980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:05:28.925 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.924+0000 7fa2e5ee5980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:28.925 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.924+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.926 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.924+0000 7fa2e5ee5980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:28.926 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.924+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.927 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.924+0000 7fa2e5ee5980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:05:28.927 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.924+0000 7fa2e5ee5980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:28.927 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.924+0000 7fa2e5ee5980 20 rados->read ofs=0 len=0 2024-03-04T18:05:28.928 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.928+0000 7fa2e5ee5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:28.929 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.928+0000 7fa199fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:28.936 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.936+0000 7fa2e5ee5980 20 remove_watcher() i=0 2024-03-04T18:05:28.936 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.936+0000 7fa2e5ee5980 2 removed watcher, disabling cache 2024-03-04T18:05:28.938 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.936+0000 7fa2e5ee5980 20 remove_watcher() i=1 2024-03-04T18:05:28.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.940+0000 7fa2e5ee5980 20 remove_watcher() i=2 2024-03-04T18:05:28.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.940+0000 7fa2e5ee5980 20 remove_watcher() i=3 2024-03-04T18:05:28.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.944+0000 7fa2e5ee5980 20 remove_watcher() i=4 2024-03-04T18:05:28.947 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.944+0000 7fa2e5ee5980 20 remove_watcher() i=5 2024-03-04T18:05:28.950 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.948+0000 7fa2e5ee5980 20 remove_watcher() i=6 2024-03-04T18:05:28.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:28.952+0000 7fa2e5ee5980 20 remove_watcher() i=7 2024-03-04T18:05:28.969 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:28.969 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:05:28.969 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-do-reshard id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:05:28.969 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:28.969 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at do_reshard 2024-03-04T18:05:29.006 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:29.006 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:29.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.048+0000 7f939288c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.254 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.252+0000 7f939288c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.254 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.252+0000 7f939288c980 20 realm 2024-03-04T18:05:29.254 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.252+0000 7f939288c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.252+0000 7f939288c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.252+0000 7f939288c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:29.255 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.252+0000 7f939288c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.256 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.252+0000 7f939288c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.256 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.252+0000 7f939288c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.258 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.256+0000 7f939288c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:29.258 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.256+0000 7f939288c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.260 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.260+0000 7f939288c980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:29.260 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.260+0000 7f939288c980 20 searching for the correct realm 2024-03-04T18:05:29.266 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.264+0000 7f938fba3640 0 -- 172.21.15.137:0/406482499 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f9384008040 msgr2=0x7f9384078a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:29.478 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.476+0000 7f938fba3640 0 -- 172.21.15.137:0/406482499 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x55b4a60a4130 msgr2=0x55b4a60a6520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:29.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.492+0000 7f939288c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:29.495 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.492+0000 7f939288c980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:05:29.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.492+0000 7f939288c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:29.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.492+0000 7f939288c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:29.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.492+0000 7f939288c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:29.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.492+0000 7f939288c980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:05:29.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.492+0000 7f939288c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.496 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.497 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:29.497 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:29.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 20 zone default found 2024-03-04T18:05:29.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 4 Realm: () 2024-03-04T18:05:29.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:05:29.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:05:29.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:29.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 20 zonegroup default 2024-03-04T18:05:29.498 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.499 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.499 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.499 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.500 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.496+0000 7f939288c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.500 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.500+0000 7f939288c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.500 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.500+0000 7f939288c980 20 started sync module instance, tier type = 2024-03-04T18:05:29.500 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.500+0000 7f939288c980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:05:29.528 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.524+0000 7f939288c980 20 add_watcher() i=0 2024-03-04T18:05:29.528 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.524+0000 7f939288c980 20 add_watcher() i=1 2024-03-04T18:05:29.528 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.524+0000 7f939288c980 20 add_watcher() i=2 2024-03-04T18:05:29.528 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.524+0000 7f939288c980 20 add_watcher() i=3 2024-03-04T18:05:29.528 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.524+0000 7f939288c980 20 add_watcher() i=4 2024-03-04T18:05:29.528 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.524+0000 7f939288c980 20 add_watcher() i=5 2024-03-04T18:05:29.531 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.528+0000 7f939288c980 20 add_watcher() i=6 2024-03-04T18:05:29.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.532+0000 7f939288c980 20 add_watcher() i=7 2024-03-04T18:05:29.533 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.532+0000 7f939288c980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:29.538 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.536+0000 7f9246fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:29.538 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.536+0000 7f939288c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:29.539 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.536+0000 7f939288c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:29.539 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.536+0000 7f939288c980 5 note: GC not initialized 2024-03-04T18:05:29.540 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.536+0000 7f9245fe3640 20 reqs_thread_entry: start 2024-03-04T18:05:29.540 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.536+0000 7f939288c980 20 init_complete bucket index max shards: 1 2024-03-04T18:05:29.540 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.536+0000 7f939288c980 20 Filter name: none 2024-03-04T18:05:29.540 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.536+0000 7f923effd640 20 reqs_thread_entry: start 2024-03-04T18:05:29.540 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.536+0000 7f939288c980 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2024-03-04T18:05:29.540 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.536+0000 7f939288c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.540+0000 7f939288c980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:29.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.540+0000 7f939288c980 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2024-03-04T18:05:29.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.540+0000 7f939288c980 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2024-03-04T18:05:29.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.540+0000 7f939288c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 : miss 2024-03-04T18:05:29.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.540+0000 7f939288c980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.540+0000 7f939288c980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:05:29.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.540+0000 7f939288c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 info.flags=0x17 2024-03-04T18:05:29.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.540+0000 7f939288c980 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 to cache LRU end 2024-03-04T18:05:29.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.540+0000 7f939288c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:05:29.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.540+0000 7f939288c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:29.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.540+0000 7f939288c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 2024-03-04T18:05:29.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.544+0000 7f939288c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 info.flags=0x17 2024-03-04T18:05:29.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.544+0000 7f939288c980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 to cache LRU end 2024-03-04T18:05:29.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.544+0000 7f939288c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:29.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.544+0000 7f939288c980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28, ofs0, ns] 2024-03-04T18:05:29.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.548+0000 7f926f7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627879 cookie 94234368204256 notifier 6476 bl.length()=945 2024-03-04T18:05:29.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.548+0000 7f926f7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 info.flags=0x17 2024-03-04T18:05:29.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.548+0000 7f926f7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 to cache LRU end 2024-03-04T18:05:29.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.548+0000 7f926f7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:29.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.548+0000 7f939288c980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): orig_sources= new_sources= 2024-03-04T18:05:29.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.548+0000 7f939288c980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): potential sources added=[] removed=[] 2024-03-04T18:05:29.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.548+0000 7f939288c980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): orig_dests= new_dests= 2024-03-04T18:05:29.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.548+0000 7f939288c980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): potential dests added=[] removed=[] 2024-03-04T18:05:29.553 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.552+0000 7f939288c980 -1 Injecting error=-5 at location=do_reshard 2024-03-04T18:05:29.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.556+0000 7f939288c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 : hit (requested=0x17, cached=0x17) 2024-03-04T18:05:29.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.556+0000 7f939288c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 2024-03-04T18:05:29.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.556+0000 7f939288c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 info.flags=0x17 2024-03-04T18:05:29.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.556+0000 7f939288c980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 to cache LRU end 2024-03-04T18:05:29.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.556+0000 7f939288c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:29.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.556+0000 7f939288c980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28, ofs0, ns] 2024-03-04T18:05:29.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.556+0000 7f926f7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627880 cookie 94234368204256 notifier 6476 bl.length()=913 2024-03-04T18:05:29.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.556+0000 7f926f7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 info.flags=0x17 2024-03-04T18:05:29.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.556+0000 7f926f7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 to cache LRU end 2024-03-04T18:05:29.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.556+0000 7f926f7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:29.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.560+0000 7f939288c980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): orig_sources= new_sources= 2024-03-04T18:05:29.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.560+0000 7f939288c980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): potential sources added=[] removed=[] 2024-03-04T18:05:29.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.560+0000 7f939288c980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): orig_dests= new_dests= 2024-03-04T18:05:29.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.560+0000 7f939288c980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): potential dests added=[] removed=[] 2024-03-04T18:05:29.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.560+0000 7f939288c980 1 execute INFO: reshard of bucket "error-at-do-reshard" canceled due to errors 2024-03-04T18:05:29.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.564+0000 7f9246fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:29.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.568+0000 7f939288c980 20 remove_watcher() i=0 2024-03-04T18:05:29.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.568+0000 7f939288c980 2 removed watcher, disabling cache 2024-03-04T18:05:29.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.568+0000 7f939288c980 20 remove_watcher() i=1 2024-03-04T18:05:29.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.572+0000 7f939288c980 20 remove_watcher() i=2 2024-03-04T18:05:29.580 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.580+0000 7f939288c980 20 remove_watcher() i=3 2024-03-04T18:05:29.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.580+0000 7f939288c980 20 remove_watcher() i=4 2024-03-04T18:05:29.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.584+0000 7f939288c980 20 remove_watcher() i=5 2024-03-04T18:05:29.587 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.584+0000 7f939288c980 20 remove_watcher() i=6 2024-03-04T18:05:29.590 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.588+0000 7f939288c980 20 remove_watcher() i=7 2024-03-04T18:05:29.605 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-03-04T18:05:29.634 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:29.634 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:29.676 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.676+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.676+0000 7f33bc434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.676+0000 7f33bc434980 20 realm 2024-03-04T18:05:29.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.676+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.676+0000 7f33bc434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.679 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.676+0000 7f33bc434980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:29.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.676+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.676+0000 7f33bc434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.676+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.680+0000 7f33bc434980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:29.680 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.680+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.680+0000 7f33bc434980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:29.681 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.680+0000 7f33bc434980 20 searching for the correct realm 2024-03-04T18:05:29.687 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.684+0000 7f33ba74d640 0 -- 172.21.15.137:0/3618159771 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f33ac0f6f70 msgr2=0x7f33ac0ac260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:05:29.900 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.900+0000 7f33ba74d640 0 -- 172.21.15.137:0/3618159771 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x56423039e910 msgr2=0x5642303a0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:29.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.908+0000 7f33bc434980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:29.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.908+0000 7f33bc434980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:05:29.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.908+0000 7f33bc434980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:29.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.908+0000 7f33bc434980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:29.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.908+0000 7f33bc434980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:29.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.908+0000 7f33bc434980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:05:29.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.908+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.908+0000 7f33bc434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.908+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.908+0000 7f33bc434980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:29.911 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.908+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:29.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 20 zone default found 2024-03-04T18:05:29.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 4 Realm: () 2024-03-04T18:05:29.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:05:29.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:05:29.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:29.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 20 zonegroup default 2024-03-04T18:05:29.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.913 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.914 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 20 started sync module instance, tier type = 2024-03-04T18:05:29.915 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.912+0000 7f33bc434980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:05:29.935 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.932+0000 7f33bc434980 20 add_watcher() i=0 2024-03-04T18:05:29.935 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.932+0000 7f33bc434980 20 add_watcher() i=1 2024-03-04T18:05:29.935 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.932+0000 7f33bc434980 20 add_watcher() i=2 2024-03-04T18:05:29.935 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.932+0000 7f33bc434980 20 add_watcher() i=3 2024-03-04T18:05:29.935 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.932+0000 7f33bc434980 20 add_watcher() i=4 2024-03-04T18:05:29.936 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.932+0000 7f33bc434980 20 add_watcher() i=5 2024-03-04T18:05:29.936 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.932+0000 7f33bc434980 20 add_watcher() i=6 2024-03-04T18:05:29.937 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.936+0000 7f33bc434980 20 add_watcher() i=7 2024-03-04T18:05:29.937 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.936+0000 7f33bc434980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:29.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.940+0000 7f326b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.940+0000 7f33bc434980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.940+0000 7f33bc434980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.940+0000 7f33bc434980 5 note: GC not initialized 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.940+0000 7f326a7fc640 20 reqs_thread_entry: start 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.940+0000 7f33bc434980 20 init_complete bucket index max shards: 1 2024-03-04T18:05:29.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.940+0000 7f33bc434980 20 Filter name: none 2024-03-04T18:05:29.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.940+0000 7f31dbfff640 20 reqs_thread_entry: start 2024-03-04T18:05:29.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.940+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.940+0000 7f33bc434980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:29.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.940+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.944+0000 7f33bc434980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:05:29.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.944+0000 7f33bc434980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:29.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.944+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.944+0000 7f33bc434980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:29.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.944+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.944+0000 7f33bc434980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:05:29.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.944+0000 7f33bc434980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:29.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.944+0000 7f33bc434980 20 rados->read ofs=0 len=0 2024-03-04T18:05:29.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.944+0000 7f33bc434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:29.947 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.944+0000 7f326b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:29.951 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.948+0000 7f33bc434980 20 remove_watcher() i=0 2024-03-04T18:05:29.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.948+0000 7f33bc434980 2 removed watcher, disabling cache 2024-03-04T18:05:29.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.952+0000 7f33bc434980 20 remove_watcher() i=1 2024-03-04T18:05:29.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.952+0000 7f33bc434980 20 remove_watcher() i=2 2024-03-04T18:05:29.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.956+0000 7f33bc434980 20 remove_watcher() i=3 2024-03-04T18:05:29.959 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.956+0000 7f33bc434980 20 remove_watcher() i=4 2024-03-04T18:05:29.962 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.960+0000 7f33bc434980 20 remove_watcher() i=5 2024-03-04T18:05:29.965 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.964+0000 7f33bc434980 20 remove_watcher() i=6 2024-03-04T18:05:29.967 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:29.964+0000 7f33bc434980 20 remove_watcher() i=7 2024-03-04T18:05:29.981 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:29.982 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:05:29.982 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-do-reshard id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:05:29.982 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:29.983 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:29.983 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:30.022 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:30.023 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:30.086 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:30.121 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:30.166 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.164+0000 7f1842afa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.164+0000 7f1842afa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.164+0000 7f1842afa980 20 realm 2024-03-04T18:05:30.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.164+0000 7f1842afa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.168+0000 7f1842afa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.169 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.168+0000 7f1842afa980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:30.169 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.168+0000 7f1842afa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.169 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.168+0000 7f1842afa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.169 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.168+0000 7f1842afa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.169 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.168+0000 7f1842afa980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:30.169 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.168+0000 7f1842afa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.168+0000 7f1842afa980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:30.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.168+0000 7f1842afa980 20 searching for the correct realm 2024-03-04T18:05:30.173 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.172+0000 7f1840612640 0 -- 172.21.15.137:0/2291132263 >> [v2:172.21.15.137:6824/1039931256,v1:172.21.15.137:6825/1039931256] conn(0x557067b8df60 msgr2=0x557067b903b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:30.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.176+0000 7f1840612640 0 -- 172.21.15.137:0/2291132263 >> [v2:172.21.15.173:6805/879938785,v1:172.21.15.173:6808/879938785] conn(0x7f18300bfbb0 msgr2=0x7f18300f4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:30.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.396+0000 7f1842afa980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:30.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.396+0000 7f1842afa980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:05:30.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.396+0000 7f1842afa980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:30.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.396+0000 7f1842afa980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:30.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.396+0000 7f1842afa980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:30.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.396+0000 7f1842afa980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:05:30.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.396+0000 7f1842afa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.398 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.396+0000 7f1842afa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.399 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.396+0000 7f1842afa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.399 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.396+0000 7f1842afa980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:30.399 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.396+0000 7f1842afa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:30.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 20 zone default found 2024-03-04T18:05:30.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 4 Realm: () 2024-03-04T18:05:30.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:05:30.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:05:30.400 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:30.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 20 zonegroup default 2024-03-04T18:05:30.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.401 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 20 started sync module instance, tier type = 2024-03-04T18:05:30.402 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.400+0000 7f1842afa980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:05:30.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.424+0000 7f1842afa980 20 add_watcher() i=0 2024-03-04T18:05:30.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.424+0000 7f1842afa980 20 add_watcher() i=1 2024-03-04T18:05:30.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.424+0000 7f1842afa980 20 add_watcher() i=2 2024-03-04T18:05:30.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.424+0000 7f1842afa980 20 add_watcher() i=3 2024-03-04T18:05:30.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.424+0000 7f1842afa980 20 add_watcher() i=4 2024-03-04T18:05:30.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.424+0000 7f1842afa980 20 add_watcher() i=5 2024-03-04T18:05:30.426 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.424+0000 7f1842afa980 20 add_watcher() i=6 2024-03-04T18:05:30.428 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.432+0000 7f1842afa980 20 add_watcher() i=7 2024-03-04T18:05:30.428 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.432+0000 7f1842afa980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:30.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.432+0000 7f16f67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:30.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.432+0000 7f1842afa980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:30.435 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.432+0000 7f1842afa980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:30.436 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.432+0000 7f1842afa980 5 note: GC not initialized 2024-03-04T18:05:30.436 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f16f57e2640 20 reqs_thread_entry: start 2024-03-04T18:05:30.437 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 20 init_complete bucket index max shards: 1 2024-03-04T18:05:30.437 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 20 Filter name: none 2024-03-04T18:05:30.437 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1666ffd640 20 reqs_thread_entry: start 2024-03-04T18:05:30.437 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2024-03-04T18:05:30.437 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:30.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2024-03-04T18:05:30.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2024-03-04T18:05:30.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 : miss 2024-03-04T18:05:30.438 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:05:30.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 info.flags=0x17 2024-03-04T18:05:30.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 to cache LRU end 2024-03-04T18:05:30.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:05:30.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:30.440 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.436+0000 7f1842afa980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 2024-03-04T18:05:30.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.444+0000 7f1842afa980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 info.flags=0x17 2024-03-04T18:05:30.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.444+0000 7f1842afa980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 to cache LRU end 2024-03-04T18:05:30.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.444+0000 7f1842afa980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:30.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.444+0000 7f1842afa980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28, ofs0, ns] 2024-03-04T18:05:30.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.444+0000 7f17177fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627881 cookie 93941264922768 notifier 6506 bl.length()=945 2024-03-04T18:05:30.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.444+0000 7f17177fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 info.flags=0x17 2024-03-04T18:05:30.445 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.444+0000 7f17177fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 to cache LRU end 2024-03-04T18:05:30.446 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.444+0000 7f17177fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:30.447 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.444+0000 7f1842afa980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): orig_sources= new_sources= 2024-03-04T18:05:30.448 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.444+0000 7f1842afa980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): potential sources added=[] removed=[] 2024-03-04T18:05:30.448 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.444+0000 7f1842afa980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): orig_dests= new_dests= 2024-03-04T18:05:30.448 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.444+0000 7f1842afa980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): potential dests added=[] removed=[] 2024-03-04T18:05:30.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.448+0000 7f1842afa980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28.1.0 2024-03-04T18:05:30.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.448+0000 7f1842afa980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28.1.0 2024-03-04T18:05:30.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.448+0000 7f1842afa980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28.1.1 2024-03-04T18:05:30.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.448+0000 7f1842afa980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28.1.1 2024-03-04T18:05:30.450 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.448+0000 7f1842afa980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28.0 2024-03-04T18:05:30.454 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.452+0000 7f1842afa980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 : hit (requested=0x17, cached=0x17) 2024-03-04T18:05:30.454 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.452+0000 7f1842afa980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 2024-03-04T18:05:30.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.452+0000 7f1842afa980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 info.flags=0x17 2024-03-04T18:05:30.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.456+0000 7f1842afa980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 to cache LRU end 2024-03-04T18:05:30.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.456+0000 7f1842afa980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:30.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.456+0000 7f1842afa980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28, ofs0, ns] 2024-03-04T18:05:30.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.456+0000 7f17177fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627882 cookie 93941264922768 notifier 6506 bl.length()=913 2024-03-04T18:05:30.456 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.456+0000 7f17177fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 info.flags=0x17 2024-03-04T18:05:30.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.456+0000 7f17177fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 to cache LRU end 2024-03-04T18:05:30.457 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.456+0000 7f17177fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:30.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.456+0000 7f1842afa980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): orig_sources= new_sources= 2024-03-04T18:05:30.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.456+0000 7f1842afa980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): potential sources added=[] removed=[] 2024-03-04T18:05:30.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.456+0000 7f1842afa980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): orig_dests= new_dests= 2024-03-04T18:05:30.458 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.456+0000 7f1842afa980 20 handle_bi_update(): bucket=:error-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28]): potential dests added=[] removed=[] 2024-03-04T18:05:30.461 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.460+0000 7f1842afa980 1 execute INFO: reshard of bucket "error-at-do-reshard" completed successfully 2024-03-04T18:05:30.464 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.464+0000 7f16f67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:30.468 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.468+0000 7f1842afa980 20 remove_watcher() i=0 2024-03-04T18:05:30.468 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.468+0000 7f1842afa980 2 removed watcher, disabling cache 2024-03-04T18:05:30.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.468+0000 7f1842afa980 20 remove_watcher() i=1 2024-03-04T18:05:30.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.472+0000 7f1842afa980 20 remove_watcher() i=2 2024-03-04T18:05:30.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.472+0000 7f1842afa980 20 remove_watcher() i=3 2024-03-04T18:05:30.477 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.476+0000 7f1842afa980 20 remove_watcher() i=4 2024-03-04T18:05:30.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.476+0000 7f1842afa980 20 remove_watcher() i=5 2024-03-04T18:05:30.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.480+0000 7f1842afa980 20 remove_watcher() i=6 2024-03-04T18:05:30.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.484+0000 7f1842afa980 20 remove_watcher() i=7 2024-03-04T18:05:30.500 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-03-04T18:05:30.527 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:30.527 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:30.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.568+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.568+0000 7ff263213980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.568+0000 7ff263213980 20 realm 2024-03-04T18:05:30.571 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.568+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.568+0000 7ff263213980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.568+0000 7ff263213980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:30.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.568+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.572+0000 7ff263213980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.572+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.573 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.572+0000 7ff263213980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:30.573 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.572+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.573 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.572+0000 7ff263213980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:30.574 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.572+0000 7ff263213980 20 searching for the correct realm 2024-03-04T18:05:30.577 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.576+0000 7ff26052a640 0 -- 172.21.15.137:0/1557518346 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7ff2580cc040 msgr2=0x7ff2580f4040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:30.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.584+0000 7ff26052a640 0 -- 172.21.15.137:0/1557518346 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x55feb12671d0 msgr2=0x55feb12695c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:30.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:30.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:05:30.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:30.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:30.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:30.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:05:30.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:30.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:30.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 zone default found 2024-03-04T18:05:30.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 4 Realm: () 2024-03-04T18:05:30.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:05:30.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:05:30.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:30.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 zonegroup default 2024-03-04T18:05:30.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.592+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.596+0000 7ff263213980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.596+0000 7ff263213980 20 started sync module instance, tier type = 2024-03-04T18:05:30.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.596+0000 7ff263213980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:05:30.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.612+0000 7ff263213980 20 add_watcher() i=0 2024-03-04T18:05:30.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.612+0000 7ff263213980 20 add_watcher() i=1 2024-03-04T18:05:30.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.612+0000 7ff263213980 20 add_watcher() i=2 2024-03-04T18:05:30.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.612+0000 7ff263213980 20 add_watcher() i=3 2024-03-04T18:05:30.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.612+0000 7ff263213980 20 add_watcher() i=4 2024-03-04T18:05:30.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.612+0000 7ff263213980 20 add_watcher() i=5 2024-03-04T18:05:30.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.612+0000 7ff263213980 20 add_watcher() i=6 2024-03-04T18:05:30.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.620+0000 7ff263213980 20 add_watcher() i=7 2024-03-04T18:05:30.618 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.620+0000 7ff263213980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:30.621 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.620+0000 7ff1177e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:30.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.620+0000 7ff263213980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:30.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.620+0000 7ff263213980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:30.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.620+0000 7ff263213980 5 note: GC not initialized 2024-03-04T18:05:30.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.620+0000 7ff1167e4640 20 reqs_thread_entry: start 2024-03-04T18:05:30.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.620+0000 7ff263213980 20 init_complete bucket index max shards: 1 2024-03-04T18:05:30.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.620+0000 7ff263213980 20 Filter name: none 2024-03-04T18:05:30.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.620+0000 7ff10f7fe640 20 reqs_thread_entry: start 2024-03-04T18:05:30.622 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.620+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.623 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.620+0000 7ff263213980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:30.623 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.620+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.624 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.624+0000 7ff263213980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:05:30.624 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.624+0000 7ff263213980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:30.624 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.624+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.625 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.624+0000 7ff263213980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:30.625 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.624+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.625 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.624+0000 7ff263213980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:05:30.625 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.624+0000 7ff263213980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:30.626 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.624+0000 7ff263213980 20 rados->read ofs=0 len=0 2024-03-04T18:05:30.626 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.624+0000 7ff263213980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:30.627 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.624+0000 7ff1177e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:30.632 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.632+0000 7ff263213980 20 remove_watcher() i=0 2024-03-04T18:05:30.632 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.632+0000 7ff263213980 2 removed watcher, disabling cache 2024-03-04T18:05:30.634 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.632+0000 7ff263213980 20 remove_watcher() i=1 2024-03-04T18:05:30.636 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.636+0000 7ff263213980 20 remove_watcher() i=2 2024-03-04T18:05:30.638 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.636+0000 7ff263213980 20 remove_watcher() i=3 2024-03-04T18:05:30.640 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.636+0000 7ff263213980 20 remove_watcher() i=4 2024-03-04T18:05:30.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.640+0000 7ff263213980 20 remove_watcher() i=5 2024-03-04T18:05:30.644 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.644+0000 7ff263213980 20 remove_watcher() i=6 2024-03-04T18:05:30.647 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:30.644+0000 7ff263213980 20 remove_watcher() i=7 2024-03-04T18:05:30.660 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:30.660 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:05:30.660 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: error-at-do-reshard id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.28 num_objects: 19 size_kb: 1 num_shards: 2 2024-03-04T18:05:30.661 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:30.662 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:30.662 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:30.681 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:30.681 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:31.025 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:31.025 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:31.198 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: reshard bucket with abort at do_reshard 2024-03-04T18:05:31.198 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:31.201 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:31.201 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:31.310 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:31.310 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:31.392 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:31.392 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:31.474 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:31.474 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:31.556 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:31.556 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:31.639 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:31.639 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:31.723 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:31.723 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:31.807 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:31.807 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:31.887 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:31.887 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:31.972 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:31.972 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:32.055 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:32.055 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:32.139 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:32.139 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:32.224 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:32.224 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:32.312 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:32.312 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:32.395 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:32.395 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:32.493 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:32.493 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:32.572 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:32.572 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:32.655 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:32.655 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:32.735 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:32.735 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:32.819 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:32.819 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:32.898 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:32.898 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:32.983 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:32.983 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:05:33.065 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-03-04T18:05:33.097 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:33.097 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:33.140 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.140+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.144+0000 7f2326ea2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.144+0000 7f2326ea2980 20 realm 2024-03-04T18:05:33.142 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.144+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.144+0000 7f2326ea2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.144+0000 7f2326ea2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:33.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.144+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.144+0000 7f2326ea2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.143 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.144+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.148+0000 7f2326ea2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:33.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.148+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.148+0000 7f2326ea2980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:33.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.148+0000 7f2326ea2980 20 searching for the correct realm 2024-03-04T18:05:33.150 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.152+0000 7f23251bb640 0 -- 172.21.15.137:0/2890996387 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55e6e2c7eae0 msgr2=0x55e6e2c80fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:33.156 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.160+0000 7f23251bb640 0 -- 172.21.15.137:0/2890996387 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x55e6e2c722e0 msgr2=0x55e6e2c74770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:05:33.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:33.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:05:33.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:33.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:33.165 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:33.166 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:05:33.166 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:33.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:33.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 zone default found 2024-03-04T18:05:33.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 4 Realm: () 2024-03-04T18:05:33.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:05:33.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:05:33.167 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:33.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 zonegroup default 2024-03-04T18:05:33.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.168 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.168+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.169 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.172+0000 7f2326ea2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.169 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.172+0000 7f2326ea2980 20 started sync module instance, tier type = 2024-03-04T18:05:33.169 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.172+0000 7f2326ea2980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:05:33.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.188+0000 7f2326ea2980 20 add_watcher() i=0 2024-03-04T18:05:33.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.188+0000 7f2326ea2980 20 add_watcher() i=1 2024-03-04T18:05:33.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.188+0000 7f2326ea2980 20 add_watcher() i=2 2024-03-04T18:05:33.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.188+0000 7f2326ea2980 20 add_watcher() i=3 2024-03-04T18:05:33.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.188+0000 7f2326ea2980 20 add_watcher() i=4 2024-03-04T18:05:33.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.188+0000 7f2326ea2980 20 add_watcher() i=5 2024-03-04T18:05:33.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.188+0000 7f2326ea2980 20 add_watcher() i=6 2024-03-04T18:05:33.190 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.192+0000 7f2326ea2980 20 add_watcher() i=7 2024-03-04T18:05:33.190 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.192+0000 7f2326ea2980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:33.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.196+0000 7f21dafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:33.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.196+0000 7f2326ea2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:33.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.196+0000 7f2326ea2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:33.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.196+0000 7f2326ea2980 5 note: GC not initialized 2024-03-04T18:05:33.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.196+0000 7f21d9fe3640 20 reqs_thread_entry: start 2024-03-04T18:05:33.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.196+0000 7f2326ea2980 20 init_complete bucket index max shards: 1 2024-03-04T18:05:33.194 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.196+0000 7f2326ea2980 20 Filter name: none 2024-03-04T18:05:33.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.196+0000 7f214b7fe640 20 reqs_thread_entry: start 2024-03-04T18:05:33.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.196+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.196+0000 7f2326ea2980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:33.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.196+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.197 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.200+0000 7f2326ea2980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:05:33.197 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.200+0000 7f2326ea2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:33.197 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.200+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.197 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.200+0000 7f2326ea2980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:33.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.200+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.200+0000 7f2326ea2980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:05:33.198 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.200+0000 7f2326ea2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:33.199 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.200+0000 7f2326ea2980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.199 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.200+0000 7f2326ea2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.201 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.204+0000 7f21dafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:33.205 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.208+0000 7f2326ea2980 20 remove_watcher() i=0 2024-03-04T18:05:33.205 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.208+0000 7f2326ea2980 2 removed watcher, disabling cache 2024-03-04T18:05:33.207 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.208+0000 7f2326ea2980 20 remove_watcher() i=1 2024-03-04T18:05:33.209 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.212+0000 7f2326ea2980 20 remove_watcher() i=2 2024-03-04T18:05:33.211 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.212+0000 7f2326ea2980 20 remove_watcher() i=3 2024-03-04T18:05:33.213 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.216+0000 7f2326ea2980 20 remove_watcher() i=4 2024-03-04T18:05:33.215 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.216+0000 7f2326ea2980 20 remove_watcher() i=5 2024-03-04T18:05:33.217 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.220+0000 7f2326ea2980 20 remove_watcher() i=6 2024-03-04T18:05:33.220 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.224+0000 7f2326ea2980 20 remove_watcher() i=7 2024-03-04T18:05:33.234 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.234 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:05:33.234 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: abort-at-do-reshard id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:05:33.234 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.235 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at do_reshard 2024-03-04T18:05:33.267 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:33.267 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:33.310 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.312+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.316+0000 7f774963a980 20 realm 2024-03-04T18:05:33.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.316+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.316+0000 7f774963a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:33.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.316+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.313 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.316+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.314 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:33.315 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.316+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:33.316 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.316+0000 7f774963a980 20 searching for the correct realm 2024-03-04T18:05:33.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.320+0000 7f7746951640 0 -- 172.21.15.137:0/3100790417 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f77381109e0 msgr2=0x7f7738001a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:05:33.531 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.532+0000 7f7746951640 0 -- 172.21.15.137:0/3100790417 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x55871b0f8580 msgr2=0x55871b0fa970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:05:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:05:33.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.548+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.548+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.548+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:33.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:33.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 20 zone default found 2024-03-04T18:05:33.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 4 Realm: () 2024-03-04T18:05:33.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:05:33.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:05:33.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:33.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 20 zonegroup default 2024-03-04T18:05:33.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 20 started sync module instance, tier type = 2024-03-04T18:05:33.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.552+0000 7f774963a980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:05:33.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=0 2024-03-04T18:05:33.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=1 2024-03-04T18:05:33.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=2 2024-03-04T18:05:33.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=3 2024-03-04T18:05:33.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=4 2024-03-04T18:05:33.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=5 2024-03-04T18:05:33.575 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=6 2024-03-04T18:05:33.577 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.580+0000 7f774963a980 20 add_watcher() i=7 2024-03-04T18:05:33.577 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.580+0000 7f774963a980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:33.581 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.584+0000 7f75fe7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:33.582 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.584+0000 7f774963a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:33.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.584+0000 7f774963a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:33.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.584+0000 7f774963a980 5 note: GC not initialized 2024-03-04T18:05:33.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.584+0000 7f75fd7e2640 20 reqs_thread_entry: start 2024-03-04T18:05:33.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.584+0000 7f774963a980 20 init_complete bucket index max shards: 1 2024-03-04T18:05:33.583 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.584+0000 7f774963a980 20 Filter name: none 2024-03-04T18:05:33.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.584+0000 7f75f67fc640 20 reqs_thread_entry: start 2024-03-04T18:05:33.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.588+0000 7f774963a980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-03-04T18:05:33.584 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.588+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.588+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:33.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.588+0000 7f774963a980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-03-04T18:05:33.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.588+0000 7f774963a980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-03-04T18:05:33.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.588+0000 7f774963a980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 : miss 2024-03-04T18:05:33.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.588+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.588+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:05:33.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.588+0000 7f774963a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 info.flags=0x17 2024-03-04T18:05:33.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.588+0000 7f774963a980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 to cache LRU end 2024-03-04T18:05:33.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.588+0000 7f774963a980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:05:33.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.588+0000 7f774963a980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:33.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.588+0000 7f774963a980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 2024-03-04T18:05:33.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.596+0000 7f774963a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 info.flags=0x17 2024-03-04T18:05:33.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.596+0000 7f774963a980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 to cache LRU end 2024-03-04T18:05:33.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.596+0000 7f774963a980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:33.594 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.596+0000 7f774963a980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29, ofs0, ns] 2024-03-04T18:05:33.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.596+0000 7f762a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627796 cookie 94038762984912 notifier 6933 bl.length()=945 2024-03-04T18:05:33.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.596+0000 7f762a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 info.flags=0x17 2024-03-04T18:05:33.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.596+0000 7f762a7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 to cache LRU end 2024-03-04T18:05:33.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.596+0000 7f762a7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:33.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.600+0000 7f774963a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): orig_sources= new_sources= 2024-03-04T18:05:33.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.600+0000 7f774963a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): potential sources added=[] removed=[] 2024-03-04T18:05:33.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.600+0000 7f774963a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): orig_dests= new_dests= 2024-03-04T18:05:33.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.600+0000 7f774963a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): potential dests added=[] removed=[] 2024-03-04T18:05:33.601 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f774963a980 time 2024-03-04T18:05:33.604827+0000 2024-03-04T18:05:33.601 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:05:33.602 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:33.602 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f774cf0fd7b] 2024-03-04T18:05:33.602 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f774cf0ff2f] 2024-03-04T18:05:33.602 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558719a10281] 2024-03-04T18:05:33.602 INFO:tasks.workunit.client.0.smithi137.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x558719a1cd5d] 2024-03-04T18:05:33.602 INFO:tasks.workunit.client.0.smithi137.stderr: 5: main() 2024-03-04T18:05:33.602 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f774bd7ad90] 2024-03-04T18:05:33.602 INFO:tasks.workunit.client.0.smithi137.stderr: 7: __libc_start_main() 2024-03-04T18:05:33.602 INFO:tasks.workunit.client.0.smithi137.stderr: 8: _start() 2024-03-04T18:05:33.602 INFO:tasks.workunit.client.0.smithi137.stderr:*** Caught signal (Aborted) ** 2024-03-04T18:05:33.602 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7f774963a980 thread_name:radosgw-admin 2024-03-04T18:05:33.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.604+0000 7f774963a980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f774963a980 time 2024-03-04T18:05:33.604827+0000 2024-03-04T18:05:33.603 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:05:33.603 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.603 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:33.603 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f774cf0fd7b] 2024-03-04T18:05:33.603 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f774cf0ff2f] 2024-03-04T18:05:33.603 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558719a10281] 2024-03-04T18:05:33.603 INFO:tasks.workunit.client.0.smithi137.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x558719a1cd5d] 2024-03-04T18:05:33.603 INFO:tasks.workunit.client.0.smithi137.stderr: 5: main() 2024-03-04T18:05:33.603 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f774bd7ad90] 2024-03-04T18:05:33.603 INFO:tasks.workunit.client.0.smithi137.stderr: 7: __libc_start_main() 2024-03-04T18:05:33.603 INFO:tasks.workunit.client.0.smithi137.stderr: 8: _start() 2024-03-04T18:05:33.603 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.614 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:33.614 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f774bd93520] 2024-03-04T18:05:33.614 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:05:33.614 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:05:33.615 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:05:33.615 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f774cf0fdd5] 2024-03-04T18:05:33.615 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f774cf0ff2f] 2024-03-04T18:05:33.615 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558719a10281] 2024-03-04T18:05:33.615 INFO:tasks.workunit.client.0.smithi137.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x558719a1cd5d] 2024-03-04T18:05:33.615 INFO:tasks.workunit.client.0.smithi137.stderr: 9: main() 2024-03-04T18:05:33.615 INFO:tasks.workunit.client.0.smithi137.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f774bd7ad90] 2024-03-04T18:05:33.615 INFO:tasks.workunit.client.0.smithi137.stderr: 11: __libc_start_main() 2024-03-04T18:05:33.615 INFO:tasks.workunit.client.0.smithi137.stderr: 12: _start() 2024-03-04T18:05:33.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.604+0000 7f774963a980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:05:33.615 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7f774963a980 thread_name:radosgw-admin 2024-03-04T18:05:33.615 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.615 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:33.615 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f774bd93520] 2024-03-04T18:05:33.616 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:05:33.616 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:05:33.616 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:05:33.616 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f774cf0fdd5] 2024-03-04T18:05:33.616 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f774cf0ff2f] 2024-03-04T18:05:33.616 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558719a10281] 2024-03-04T18:05:33.616 INFO:tasks.workunit.client.0.smithi137.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x558719a1cd5d] 2024-03-04T18:05:33.616 INFO:tasks.workunit.client.0.smithi137.stderr: 9: main() 2024-03-04T18:05:33.616 INFO:tasks.workunit.client.0.smithi137.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f774bd7ad90] 2024-03-04T18:05:33.616 INFO:tasks.workunit.client.0.smithi137.stderr: 11: __libc_start_main() 2024-03-04T18:05:33.616 INFO:tasks.workunit.client.0.smithi137.stderr: 12: _start() 2024-03-04T18:05:33.616 INFO:tasks.workunit.client.0.smithi137.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:05:33.616 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.616 INFO:tasks.workunit.client.0.smithi137.stderr:--- begin dump of recent events --- 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr: -499> 2024-03-04T18:05:33.292+0000 7f7747152640 1 monclient: found mon.noname-c 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr: -498> 2024-03-04T18:05:33.292+0000 7f7747152640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.137:3301/0 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr: -497> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr: -496> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr: -495> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: dump: 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.617 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.618 INFO:tasks.workunit.client.0.smithi137.stderr: -494> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: _finish_auth 0 2024-03-04T18:05:33.618 INFO:tasks.workunit.client.0.smithi137.stderr: -493> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: _check_auth_tickets 2024-03-04T18:05:33.618 INFO:tasks.workunit.client.0.smithi137.stderr: -492> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:33.618 INFO:tasks.workunit.client.0.smithi137.stderr: -491> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.618 INFO:tasks.workunit.client.0.smithi137.stderr: -490> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:05:33.618 INFO:tasks.workunit.client.0.smithi137.stderr: -489> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: dump: 2024-03-04T18:05:33.618 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.618 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.618 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.618 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.618 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.618 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.618 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.618 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr: -488> 2024-03-04T18:05:33.292+0000 7f774963a980 5 monclient: authenticate success, global_id 6542 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr: -487> 2024-03-04T18:05:33.292+0000 7f774963a980 10 monclient: get_monmap_and_config success 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr: -486> 2024-03-04T18:05:33.292+0000 7f774963a980 4 set_mon_vals no callback set 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr: -485> 2024-03-04T18:05:33.292+0000 7f774963a980 10 monclient: shutdown 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr: -484> 2024-03-04T18:05:33.292+0000 7f774963a980 5 asok(0x55871ae20130) unregister_commands rotate-key 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr: -483> 2024-03-04T18:05:33.292+0000 7f774963a980 10 monclient: build_initial_monmap 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr: -482> 2024-03-04T18:05:33.292+0000 7f774963a980 1 build_initial for_mkfs: 0 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr: -481> 2024-03-04T18:05:33.292+0000 7f774963a980 10 monclient: monmap: 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:05:33.619 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:05:33.620 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.620 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:05:33.620 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:05:33.620 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:05:33.620 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.620 INFO:tasks.workunit.client.0.smithi137.stderr: -480> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding auth protocol: cephx 2024-03-04T18:05:33.620 INFO:tasks.workunit.client.0.smithi137.stderr: -479> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding auth protocol: cephx 2024-03-04T18:05:33.620 INFO:tasks.workunit.client.0.smithi137.stderr: -478> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding auth protocol: cephx 2024-03-04T18:05:33.620 INFO:tasks.workunit.client.0.smithi137.stderr: -477> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: secure 2024-03-04T18:05:33.620 INFO:tasks.workunit.client.0.smithi137.stderr: -476> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: crc 2024-03-04T18:05:33.620 INFO:tasks.workunit.client.0.smithi137.stderr: -475> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: secure 2024-03-04T18:05:33.620 INFO:tasks.workunit.client.0.smithi137.stderr: -474> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: crc 2024-03-04T18:05:33.620 INFO:tasks.workunit.client.0.smithi137.stderr: -473> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: secure 2024-03-04T18:05:33.620 INFO:tasks.workunit.client.0.smithi137.stderr: -472> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: crc 2024-03-04T18:05:33.621 INFO:tasks.workunit.client.0.smithi137.stderr: -471> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: crc 2024-03-04T18:05:33.621 INFO:tasks.workunit.client.0.smithi137.stderr: -470> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: secure 2024-03-04T18:05:33.621 INFO:tasks.workunit.client.0.smithi137.stderr: -469> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: crc 2024-03-04T18:05:33.621 INFO:tasks.workunit.client.0.smithi137.stderr: -468> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: secure 2024-03-04T18:05:33.621 INFO:tasks.workunit.client.0.smithi137.stderr: -467> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: crc 2024-03-04T18:05:33.621 INFO:tasks.workunit.client.0.smithi137.stderr: -466> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: secure 2024-03-04T18:05:33.621 INFO:tasks.workunit.client.0.smithi137.stderr: -465> 2024-03-04T18:05:33.296+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.621 INFO:tasks.workunit.client.0.smithi137.stderr: -464> 2024-03-04T18:05:33.296+0000 7f774963a980 1 librados: starting msgr at 2024-03-04T18:05:33.621 INFO:tasks.workunit.client.0.smithi137.stderr: -463> 2024-03-04T18:05:33.296+0000 7f774963a980 1 librados: starting objecter 2024-03-04T18:05:33.621 INFO:tasks.workunit.client.0.smithi137.stderr: -462> 2024-03-04T18:05:33.296+0000 7f774963a980 5 asok(0x55871ae20130) register_command objecter_requests hook 0x55871adbe4c0 2024-03-04T18:05:33.621 INFO:tasks.workunit.client.0.smithi137.stderr: -461> 2024-03-04T18:05:33.296+0000 7f774963a980 1 librados: setting wanted keys 2024-03-04T18:05:33.621 INFO:tasks.workunit.client.0.smithi137.stderr: -460> 2024-03-04T18:05:33.296+0000 7f774963a980 1 librados: calling monclient init 2024-03-04T18:05:33.621 INFO:tasks.workunit.client.0.smithi137.stderr: -459> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient: init 2024-03-04T18:05:33.621 INFO:tasks.workunit.client.0.smithi137.stderr: -458> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding auth protocol: cephx 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -457> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding auth protocol: cephx 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -456> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding auth protocol: cephx 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -455> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: secure 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -454> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: crc 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -453> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: secure 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -452> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: crc 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -451> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: secure 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -450> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: crc 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -449> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: crc 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -448> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: secure 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -447> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: crc 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -446> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: secure 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -445> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: crc 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -444> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: secure 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -443> 2024-03-04T18:05:33.296+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.622 INFO:tasks.workunit.client.0.smithi137.stderr: -442> 2024-03-04T18:05:33.296+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -441> 2024-03-04T18:05:33.296+0000 7f774963a980 5 asok(0x55871ae20130) register_command rotate-key hook 0x55871b0076c8 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -440> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -439> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient: _add_conns ranks=[0,1,2] 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -438> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): picked mon.noname-a con 0x55871aeb6370 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -437> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): picked mon.noname-b con 0x55871aec1280 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -436> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): picked mon.noname-c con 0x55871af4f400 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -435> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -434> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -433> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -432> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -431> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): authenticate will time out at 2566.412932s 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -430> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): get_auth_request con 0x55871aeb6370 auth_method 0 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -429> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -428> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.623 INFO:tasks.workunit.client.0.smithi137.stderr: -427> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.624 INFO:tasks.workunit.client.0.smithi137.stderr: -426> 2024-03-04T18:05:33.296+0000 7f7747953640 10 monclient(hunting): get_auth_request con 0x55871af4f400 auth_method 0 2024-03-04T18:05:33.624 INFO:tasks.workunit.client.0.smithi137.stderr: -425> 2024-03-04T18:05:33.296+0000 7f7747953640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.624 INFO:tasks.workunit.client.0.smithi137.stderr: -424> 2024-03-04T18:05:33.296+0000 7f7747953640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.624 INFO:tasks.workunit.client.0.smithi137.stderr: -423> 2024-03-04T18:05:33.296+0000 7f7747953640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.624 INFO:tasks.workunit.client.0.smithi137.stderr: -422> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.624 INFO:tasks.workunit.client.0.smithi137.stderr: -421> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.624 INFO:tasks.workunit.client.0.smithi137.stderr: -420> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.624 INFO:tasks.workunit.client.0.smithi137.stderr: -419> 2024-03-04T18:05:33.296+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.624 INFO:tasks.workunit.client.0.smithi137.stderr: -418> 2024-03-04T18:05:33.296+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.624 INFO:tasks.workunit.client.0.smithi137.stderr: -417> 2024-03-04T18:05:33.296+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.624 INFO:tasks.workunit.client.0.smithi137.stderr: -416> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): handle_auth_done global_id 6924 payload 824 2024-03-04T18:05:33.624 INFO:tasks.workunit.client.0.smithi137.stderr: -415> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient: _finish_hunting 0 2024-03-04T18:05:33.624 INFO:tasks.workunit.client.0.smithi137.stderr: -414> 2024-03-04T18:05:33.296+0000 7f7747152640 1 monclient: found mon.noname-a 2024-03-04T18:05:33.624 INFO:tasks.workunit.client.0.smithi137.stderr: -413> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.625 INFO:tasks.workunit.client.0.smithi137.stderr: -412> 2024-03-04T18:05:33.296+0000 7f7746951640 10 monclient: get_auth_request con 0x55871aec1280 auth_method 0 2024-03-04T18:05:33.625 INFO:tasks.workunit.client.0.smithi137.stderr: -411> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.625 INFO:tasks.workunit.client.0.smithi137.stderr: -410> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:05:33.625 INFO:tasks.workunit.client.0.smithi137.stderr: -409> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: dump: 2024-03-04T18:05:33.625 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.625 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.625 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.625 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.625 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.625 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.625 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.625 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.625 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.625 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr: -408> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: _finish_auth 0 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr: -407> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: _check_auth_tickets 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr: -406> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr: -405> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr: -404> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr: -403> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: dump: 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.626 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -402> 2024-03-04T18:05:33.296+0000 7f774963a980 5 monclient: authenticate success, global_id 6924 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -401> 2024-03-04T18:05:33.296+0000 7f763560a640 4 set_mon_vals no callback set 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -400> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient: _renew_subs 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -399> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -398> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient: _renew_subs 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -397> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -396> 2024-03-04T18:05:33.296+0000 7f774963a980 1 librados: init done 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -395> 2024-03-04T18:05:33.296+0000 7f7633e07640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -394> 2024-03-04T18:05:33.296+0000 7f7633e07640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -393> 2024-03-04T18:05:33.296+0000 7f7633e07640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -392> 2024-03-04T18:05:33.300+0000 7f7746951640 10 monclient: get_auth_request con 0x7f762c065940 auth_method 0 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -391> 2024-03-04T18:05:33.300+0000 7f7747953640 10 monclient: get_auth_request con 0x55871b034c70 auth_method 0 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -390> 2024-03-04T18:05:33.300+0000 7f7747152640 10 monclient: get_auth_request con 0x55871b052860 auth_method 0 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -389> 2024-03-04T18:05:33.300+0000 7f7746951640 10 monclient: get_auth_request con 0x55871b056a50 auth_method 0 2024-03-04T18:05:33.627 INFO:tasks.workunit.client.0.smithi137.stderr: -388> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: build_initial_monmap 2024-03-04T18:05:33.628 INFO:tasks.workunit.client.0.smithi137.stderr: -387> 2024-03-04T18:05:33.304+0000 7f774963a980 1 build_initial for_mkfs: 0 2024-03-04T18:05:33.628 INFO:tasks.workunit.client.0.smithi137.stderr: -386> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: monmap: 2024-03-04T18:05:33.628 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:05:33.628 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:05:33.628 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:05:33.628 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:05:33.628 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:05:33.628 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.628 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:05:33.628 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:05:33.628 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:05:33.628 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.628 INFO:tasks.workunit.client.0.smithi137.stderr: -385> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding auth protocol: cephx 2024-03-04T18:05:33.628 INFO:tasks.workunit.client.0.smithi137.stderr: -384> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding auth protocol: cephx 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -383> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding auth protocol: cephx 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -382> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: secure 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -381> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: crc 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -380> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: secure 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -379> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: crc 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -378> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: secure 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -377> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: crc 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -376> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: crc 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -375> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: secure 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -374> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: crc 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -373> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: secure 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -372> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: crc 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -371> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: secure 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -370> 2024-03-04T18:05:33.304+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.629 INFO:tasks.workunit.client.0.smithi137.stderr: -369> 2024-03-04T18:05:33.304+0000 7f774963a980 5 asok(0x55871ae20130) register_command objecter_requests cmddesc objecter_requests hook 0x55871b060cb0 EEXIST 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -368> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: init 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -367> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding auth protocol: cephx 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -366> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding auth protocol: cephx 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -365> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding auth protocol: cephx 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -364> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: secure 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -363> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: crc 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -362> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: secure 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -361> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: crc 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -360> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: secure 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -359> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: crc 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -358> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: crc 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -357> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: secure 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -356> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: crc 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -355> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: secure 2024-03-04T18:05:33.630 INFO:tasks.workunit.client.0.smithi137.stderr: -354> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: crc 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -353> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: secure 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -352> 2024-03-04T18:05:33.304+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -351> 2024-03-04T18:05:33.304+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -350> 2024-03-04T18:05:33.304+0000 7f774963a980 5 asok(0x55871ae20130) register_command rotate-key cmddesc rotate-key hook 0x55871b059db0 EEXIST 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -349> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -348> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: _add_conns ranks=[1,2,0] 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -347> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): picked mon.noname-b con 0x55871b085910 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -346> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): picked mon.noname-c con 0x55871b085cf0 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -345> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): picked mon.noname-a con 0x55871b080eb0 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -344> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -343> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -342> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -341> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -340> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): authenticate will time out at 2566.421309s 2024-03-04T18:05:33.631 INFO:tasks.workunit.client.0.smithi137.stderr: -339> 2024-03-04T18:05:33.304+0000 7f7746951640 10 monclient(hunting): get_auth_request con 0x55871b085cf0 auth_method 0 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -338> 2024-03-04T18:05:33.304+0000 7f7746951640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -337> 2024-03-04T18:05:33.304+0000 7f7746951640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -336> 2024-03-04T18:05:33.304+0000 7f7746951640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -335> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): get_auth_request con 0x55871b080eb0 auth_method 0 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -334> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -333> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -332> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -331> 2024-03-04T18:05:33.304+0000 7f7746951640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -330> 2024-03-04T18:05:33.304+0000 7f7746951640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -329> 2024-03-04T18:05:33.304+0000 7f7746951640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -328> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -327> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -326> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -325> 2024-03-04T18:05:33.304+0000 7f7747152640 10 monclient(hunting): get_auth_request con 0x55871b085910 auth_method 0 2024-03-04T18:05:33.632 INFO:tasks.workunit.client.0.smithi137.stderr: -324> 2024-03-04T18:05:33.304+0000 7f7747152640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr: -323> 2024-03-04T18:05:33.304+0000 7f7747152640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr: -322> 2024-03-04T18:05:33.304+0000 7f7747152640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr: -321> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): handle_auth_done global_id 6927 payload 824 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr: -320> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient: _finish_hunting 0 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr: -319> 2024-03-04T18:05:33.304+0000 7f7747953640 1 monclient: found mon.noname-a 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr: -318> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr: -317> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr: -316> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr: -315> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: dump: 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.633 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr: -314> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: _finish_auth 0 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr: -313> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: _check_auth_tickets 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr: -312> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr: -311> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr: -310> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr: -309> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: dump: 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.634 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr: -308> 2024-03-04T18:05:33.304+0000 7f774963a980 5 monclient: authenticate success, global_id 6927 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr: -307> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: _renew_subs 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr: -306> 2024-03-04T18:05:33.304+0000 7f7746150640 4 set_mon_vals no callback set 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr: -305> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr: -304> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: _renew_subs 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr: -303> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr: -302> 2024-03-04T18:05:33.308+0000 7f7631602640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr: -301> 2024-03-04T18:05:33.308+0000 7f7631602640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr: -300> 2024-03-04T18:05:33.308+0000 7f7631602640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:05:33.635 INFO:tasks.workunit.client.0.smithi137.stderr: -299> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: get_monmap_and_config 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr: -298> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: build_initial_monmap 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr: -297> 2024-03-04T18:05:33.308+0000 7f774963a980 1 build_initial for_mkfs: 0 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr: -296> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: monmap: 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr: -295> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding auth protocol: cephx 2024-03-04T18:05:33.636 INFO:tasks.workunit.client.0.smithi137.stderr: -294> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding auth protocol: cephx 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -293> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding auth protocol: cephx 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -292> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -291> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -290> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -289> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -288> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -287> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -286> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -285> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -284> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -283> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -282> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -281> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -280> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient: get_auth_request con 0x7f7620065870 auth_method 0 2024-03-04T18:05:33.637 INFO:tasks.workunit.client.0.smithi137.stderr: -279> 2024-03-04T18:05:33.308+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.638 INFO:tasks.workunit.client.0.smithi137.stderr: -278> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: init 2024-03-04T18:05:33.638 INFO:tasks.workunit.client.0.smithi137.stderr: -277> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding auth protocol: cephx 2024-03-04T18:05:33.638 INFO:tasks.workunit.client.0.smithi137.stderr: -276> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding auth protocol: cephx 2024-03-04T18:05:33.638 INFO:tasks.workunit.client.0.smithi137.stderr: -275> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding auth protocol: cephx 2024-03-04T18:05:33.638 INFO:tasks.workunit.client.0.smithi137.stderr: -274> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: secure 2024-03-04T18:05:33.638 INFO:tasks.workunit.client.0.smithi137.stderr: -273> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: crc 2024-03-04T18:05:33.638 INFO:tasks.workunit.client.0.smithi137.stderr: -272> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: secure 2024-03-04T18:05:33.638 INFO:tasks.workunit.client.0.smithi137.stderr: -271> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: crc 2024-03-04T18:05:33.638 INFO:tasks.workunit.client.0.smithi137.stderr: -270> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: secure 2024-03-04T18:05:33.638 INFO:tasks.workunit.client.0.smithi137.stderr: -269> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: crc 2024-03-04T18:05:33.638 INFO:tasks.workunit.client.0.smithi137.stderr: -268> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: crc 2024-03-04T18:05:33.638 INFO:tasks.workunit.client.0.smithi137.stderr: -267> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: secure 2024-03-04T18:05:33.638 INFO:tasks.workunit.client.0.smithi137.stderr: -266> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: crc 2024-03-04T18:05:33.638 INFO:tasks.workunit.client.0.smithi137.stderr: -265> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: secure 2024-03-04T18:05:33.639 INFO:tasks.workunit.client.0.smithi137.stderr: -264> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: crc 2024-03-04T18:05:33.639 INFO:tasks.workunit.client.0.smithi137.stderr: -263> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: secure 2024-03-04T18:05:33.639 INFO:tasks.workunit.client.0.smithi137.stderr: -262> 2024-03-04T18:05:33.308+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.639 INFO:tasks.workunit.client.0.smithi137.stderr: -261> 2024-03-04T18:05:33.308+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.639 INFO:tasks.workunit.client.0.smithi137.stderr: -260> 2024-03-04T18:05:33.308+0000 7f774963a980 5 asok(0x55871ae20130) register_command rotate-key cmddesc rotate-key hook 0x7ffe8148f828 EEXIST 2024-03-04T18:05:33.639 INFO:tasks.workunit.client.0.smithi137.stderr: -259> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:33.639 INFO:tasks.workunit.client.0.smithi137.stderr: -258> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: _add_conns ranks=[0,1,2] 2024-03-04T18:05:33.639 INFO:tasks.workunit.client.0.smithi137.stderr: -257> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): picked mon.noname-a con 0x55871b0d7b60 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:05:33.639 INFO:tasks.workunit.client.0.smithi137.stderr: -256> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): picked mon.noname-b con 0x55871b0d7f40 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:05:33.639 INFO:tasks.workunit.client.0.smithi137.stderr: -255> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): picked mon.noname-c con 0x55871b0dd360 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:05:33.639 INFO:tasks.workunit.client.0.smithi137.stderr: -254> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.639 INFO:tasks.workunit.client.0.smithi137.stderr: -253> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.639 INFO:tasks.workunit.client.0.smithi137.stderr: -252> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.639 INFO:tasks.workunit.client.0.smithi137.stderr: -251> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:33.640 INFO:tasks.workunit.client.0.smithi137.stderr: -250> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): authenticate will time out at 2566.425414s 2024-03-04T18:05:33.640 INFO:tasks.workunit.client.0.smithi137.stderr: -249> 2024-03-04T18:05:33.308+0000 7f7747953640 10 monclient(hunting): get_auth_request con 0x55871b0dd360 auth_method 0 2024-03-04T18:05:33.640 INFO:tasks.workunit.client.0.smithi137.stderr: -248> 2024-03-04T18:05:33.308+0000 7f7747953640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.640 INFO:tasks.workunit.client.0.smithi137.stderr: -247> 2024-03-04T18:05:33.308+0000 7f7747953640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.640 INFO:tasks.workunit.client.0.smithi137.stderr: -246> 2024-03-04T18:05:33.308+0000 7f7747953640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.640 INFO:tasks.workunit.client.0.smithi137.stderr: -245> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): get_auth_request con 0x55871b0d7b60 auth_method 0 2024-03-04T18:05:33.640 INFO:tasks.workunit.client.0.smithi137.stderr: -244> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.640 INFO:tasks.workunit.client.0.smithi137.stderr: -243> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.640 INFO:tasks.workunit.client.0.smithi137.stderr: -242> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.640 INFO:tasks.workunit.client.0.smithi137.stderr: -241> 2024-03-04T18:05:33.308+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.640 INFO:tasks.workunit.client.0.smithi137.stderr: -240> 2024-03-04T18:05:33.308+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.640 INFO:tasks.workunit.client.0.smithi137.stderr: -239> 2024-03-04T18:05:33.308+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.640 INFO:tasks.workunit.client.0.smithi137.stderr: -238> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.640 INFO:tasks.workunit.client.0.smithi137.stderr: -237> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr: -236> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr: -235> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): handle_auth_done global_id 6930 payload 274 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr: -234> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient: _finish_hunting 0 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr: -233> 2024-03-04T18:05:33.308+0000 7f7747152640 1 monclient: found mon.noname-a 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr: -232> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr: -231> 2024-03-04T18:05:33.308+0000 7f7746951640 10 monclient: get_auth_request con 0x55871b0d7f40 auth_method 0 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr: -230> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr: -229> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr: -228> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: dump: 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.641 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.642 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.642 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.642 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.642 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.642 INFO:tasks.workunit.client.0.smithi137.stderr: -227> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: _finish_auth 0 2024-03-04T18:05:33.642 INFO:tasks.workunit.client.0.smithi137.stderr: -226> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: _check_auth_tickets 2024-03-04T18:05:33.642 INFO:tasks.workunit.client.0.smithi137.stderr: -225> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:33.642 INFO:tasks.workunit.client.0.smithi137.stderr: -224> 2024-03-04T18:05:33.308+0000 7f774963a980 5 monclient: authenticate success, global_id 6930 2024-03-04T18:05:33.642 INFO:tasks.workunit.client.0.smithi137.stderr: -223> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: get_monmap_and_config success 2024-03-04T18:05:33.642 INFO:tasks.workunit.client.0.smithi137.stderr: -222> 2024-03-04T18:05:33.308+0000 7f774963a980 4 set_mon_vals no callback set 2024-03-04T18:05:33.642 INFO:tasks.workunit.client.0.smithi137.stderr: -221> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.642 INFO:tasks.workunit.client.0.smithi137.stderr: -220> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:05:33.642 INFO:tasks.workunit.client.0.smithi137.stderr: -219> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: dump: 2024-03-04T18:05:33.642 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr: -218> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: shutdown 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr: -217> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: build_initial_monmap 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr: -216> 2024-03-04T18:05:33.308+0000 7f774963a980 1 build_initial for_mkfs: 0 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr: -215> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: monmap: 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:05:33.643 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:05:33.644 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:05:33.644 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:05:33.644 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:05:33.644 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.644 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:05:33.644 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:05:33.644 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:05:33.644 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.644 INFO:tasks.workunit.client.0.smithi137.stderr: -214> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding auth protocol: cephx 2024-03-04T18:05:33.644 INFO:tasks.workunit.client.0.smithi137.stderr: -213> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding auth protocol: cephx 2024-03-04T18:05:33.644 INFO:tasks.workunit.client.0.smithi137.stderr: -212> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding auth protocol: cephx 2024-03-04T18:05:33.644 INFO:tasks.workunit.client.0.smithi137.stderr: -211> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.644 INFO:tasks.workunit.client.0.smithi137.stderr: -210> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.644 INFO:tasks.workunit.client.0.smithi137.stderr: -209> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -208> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -207> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -206> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -205> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -204> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -203> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -202> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -201> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -200> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -199> 2024-03-04T18:05:33.308+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -198> 2024-03-04T18:05:33.308+0000 7f774963a980 1 librados: starting msgr at 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -197> 2024-03-04T18:05:33.308+0000 7f774963a980 1 librados: starting objecter 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -196> 2024-03-04T18:05:33.308+0000 7f774963a980 5 asok(0x55871ae20130) register_command objecter_requests cmddesc objecter_requests hook 0x55871b0dcd90 EEXIST 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -195> 2024-03-04T18:05:33.308+0000 7f774963a980 1 librados: setting wanted keys 2024-03-04T18:05:33.645 INFO:tasks.workunit.client.0.smithi137.stderr: -194> 2024-03-04T18:05:33.308+0000 7f774963a980 1 librados: calling monclient init 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -193> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: init 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -192> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding auth protocol: cephx 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -191> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding auth protocol: cephx 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -190> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding auth protocol: cephx 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -189> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: secure 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -188> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: crc 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -187> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: secure 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -186> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: crc 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -185> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: secure 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -184> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: crc 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -183> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: crc 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -182> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: secure 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -181> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: crc 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -180> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: secure 2024-03-04T18:05:33.646 INFO:tasks.workunit.client.0.smithi137.stderr: -179> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: crc 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -178> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: secure 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -177> 2024-03-04T18:05:33.308+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -176> 2024-03-04T18:05:33.308+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -175> 2024-03-04T18:05:33.308+0000 7f774963a980 5 asok(0x55871ae20130) register_command rotate-key cmddesc rotate-key hook 0x55871b0a2778 EEXIST 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -174> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -173> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: _add_conns ranks=[0,1,2] 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -172> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): picked mon.noname-a con 0x55871b0d7b60 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -171> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): picked mon.noname-b con 0x55871b0dd360 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -170> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): picked mon.noname-c con 0x55871b0dad40 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -169> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -168> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -167> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -166> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -165> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): authenticate will time out at 2566.427960s 2024-03-04T18:05:33.647 INFO:tasks.workunit.client.0.smithi137.stderr: -164> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): get_auth_request con 0x55871b0d7b60 auth_method 0 2024-03-04T18:05:33.648 INFO:tasks.workunit.client.0.smithi137.stderr: -163> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.648 INFO:tasks.workunit.client.0.smithi137.stderr: -162> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.648 INFO:tasks.workunit.client.0.smithi137.stderr: -161> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.648 INFO:tasks.workunit.client.0.smithi137.stderr: -160> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.648 INFO:tasks.workunit.client.0.smithi137.stderr: -159> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.648 INFO:tasks.workunit.client.0.smithi137.stderr: -158> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.648 INFO:tasks.workunit.client.0.smithi137.stderr: -157> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient(hunting): get_auth_request con 0x55871b0dad40 auth_method 0 2024-03-04T18:05:33.648 INFO:tasks.workunit.client.0.smithi137.stderr: -156> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.648 INFO:tasks.workunit.client.0.smithi137.stderr: -155> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.648 INFO:tasks.workunit.client.0.smithi137.stderr: -154> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.648 INFO:tasks.workunit.client.0.smithi137.stderr: -153> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.648 INFO:tasks.workunit.client.0.smithi137.stderr: -152> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.648 INFO:tasks.workunit.client.0.smithi137.stderr: -151> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.648 INFO:tasks.workunit.client.0.smithi137.stderr: -150> 2024-03-04T18:05:33.312+0000 7f7746951640 10 monclient(hunting): get_auth_request con 0x55871b0dd360 auth_method 0 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr: -149> 2024-03-04T18:05:33.312+0000 7f7746951640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr: -148> 2024-03-04T18:05:33.312+0000 7f7746951640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr: -147> 2024-03-04T18:05:33.312+0000 7f7746951640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr: -146> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): handle_auth_done global_id 6933 payload 824 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr: -145> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient: _finish_hunting 0 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr: -144> 2024-03-04T18:05:33.312+0000 7f7747152640 1 monclient: found mon.noname-a 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr: -143> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr: -142> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr: -141> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr: -140> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: dump: 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.649 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr: -139> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: _finish_auth 0 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr: -138> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: _check_auth_tickets 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr: -137> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr: -136> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr: -135> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr: -134> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: dump: 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.650 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr: -133> 2024-03-04T18:05:33.312+0000 7f774963a980 5 monclient: authenticate success, global_id 6933 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr: -132> 2024-03-04T18:05:33.312+0000 7f774963a980 10 monclient: _renew_subs 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr: -131> 2024-03-04T18:05:33.312+0000 7f762affd640 4 set_mon_vals no callback set 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr: -130> 2024-03-04T18:05:33.312+0000 7f774963a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr: -129> 2024-03-04T18:05:33.312+0000 7f774963a980 10 monclient: _renew_subs 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr: -128> 2024-03-04T18:05:33.312+0000 7f774963a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr: -127> 2024-03-04T18:05:33.312+0000 7f774963a980 1 librados: init done 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr: -126> 2024-03-04T18:05:33.312+0000 7f774963a980 5 asok(0x55871ae20130) register_command cr dump hook 0x55871b0c9ee8 2024-03-04T18:05:33.651 INFO:tasks.workunit.client.0.smithi137.stderr: -125> 2024-03-04T18:05:33.312+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.652 INFO:tasks.workunit.client.0.smithi137.stderr: -124> 2024-03-04T18:05:33.312+0000 7f76297fa640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:05:33.652 INFO:tasks.workunit.client.0.smithi137.stderr: -123> 2024-03-04T18:05:33.312+0000 7f76297fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:05:33.652 INFO:tasks.workunit.client.0.smithi137.stderr: -122> 2024-03-04T18:05:33.312+0000 7f76297fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:05:33.652 INFO:tasks.workunit.client.0.smithi137.stderr: -121> 2024-03-04T18:05:33.312+0000 7f7746951640 10 monclient: get_auth_request con 0x7f7610065940 auth_method 0 2024-03-04T18:05:33.652 INFO:tasks.workunit.client.0.smithi137.stderr: -120> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient: get_auth_request con 0x55871b0f0e20 auth_method 0 2024-03-04T18:05:33.652 INFO:tasks.workunit.client.0.smithi137.stderr: -119> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.652 INFO:tasks.workunit.client.0.smithi137.stderr: -118> 2024-03-04T18:05:33.316+0000 7f774963a980 20 realm 2024-03-04T18:05:33.652 INFO:tasks.workunit.client.0.smithi137.stderr: -117> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.652 INFO:tasks.workunit.client.0.smithi137.stderr: -116> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.652 INFO:tasks.workunit.client.0.smithi137.stderr: -115> 2024-03-04T18:05:33.316+0000 7f774963a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:33.652 INFO:tasks.workunit.client.0.smithi137.stderr: -114> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.652 INFO:tasks.workunit.client.0.smithi137.stderr: -113> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.652 INFO:tasks.workunit.client.0.smithi137.stderr: -112> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.652 INFO:tasks.workunit.client.0.smithi137.stderr: -111> 2024-03-04T18:05:33.316+0000 7f7747152640 10 monclient: get_auth_request con 0x55871b0f5030 auth_method 0 2024-03-04T18:05:33.653 INFO:tasks.workunit.client.0.smithi137.stderr: -110> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:33.653 INFO:tasks.workunit.client.0.smithi137.stderr: -109> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.653 INFO:tasks.workunit.client.0.smithi137.stderr: -108> 2024-03-04T18:05:33.316+0000 7f7746951640 10 monclient: get_auth_request con 0x55871b0f8580 auth_method 0 2024-03-04T18:05:33.653 INFO:tasks.workunit.client.0.smithi137.stderr: -107> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:33.653 INFO:tasks.workunit.client.0.smithi137.stderr: -106> 2024-03-04T18:05:33.316+0000 7f774963a980 20 searching for the correct realm 2024-03-04T18:05:33.653 INFO:tasks.workunit.client.0.smithi137.stderr: -105> 2024-03-04T18:05:33.320+0000 7f7747953640 10 monclient: get_auth_request con 0x55871b0fd210 auth_method 0 2024-03-04T18:05:33.653 INFO:tasks.workunit.client.0.smithi137.stderr: -104> 2024-03-04T18:05:33.320+0000 7f7747152640 10 monclient: get_auth_request con 0x7f7740057ca0 auth_method 0 2024-03-04T18:05:33.653 INFO:tasks.workunit.client.0.smithi137.stderr: -103> 2024-03-04T18:05:33.320+0000 7f7746951640 10 monclient: get_auth_request con 0x7f77381109e0 auth_method 0 2024-03-04T18:05:33.653 INFO:tasks.workunit.client.0.smithi137.stderr: -102> 2024-03-04T18:05:33.320+0000 7f7746951640 0 -- 172.21.15.137:0/3100790417 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f77381109e0 msgr2=0x7f7738001a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:05:33.653 INFO:tasks.workunit.client.0.smithi137.stderr: -101> 2024-03-04T18:05:33.524+0000 7f7746951640 10 monclient: get_auth_request con 0x7f77381109e0 auth_method 0 2024-03-04T18:05:33.653 INFO:tasks.workunit.client.0.smithi137.stderr: -100> 2024-03-04T18:05:33.524+0000 7f7747953640 10 monclient: get_auth_request con 0x7f774005fc10 auth_method 0 2024-03-04T18:05:33.653 INFO:tasks.workunit.client.0.smithi137.stderr: -99> 2024-03-04T18:05:33.532+0000 7f7747152640 10 monclient: get_auth_request con 0x7f77380c2ba0 auth_method 0 2024-03-04T18:05:33.654 INFO:tasks.workunit.client.0.smithi137.stderr: -98> 2024-03-04T18:05:33.532+0000 7f7746951640 0 -- 172.21.15.137:0/3100790417 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x55871b0f8580 msgr2=0x55871b0fa970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:33.654 INFO:tasks.workunit.client.0.smithi137.stderr: -97> 2024-03-04T18:05:33.532+0000 7f76297fa640 1 client.6933.objecter ms_handle_reset 0x55871b0f8580 session 0x55871b0f4330 osd.3 2024-03-04T18:05:33.654 INFO:tasks.workunit.client.0.smithi137.stderr: -96> 2024-03-04T18:05:33.532+0000 7f76297fa640 10 monclient: _renew_subs 2024-03-04T18:05:33.654 INFO:tasks.workunit.client.0.smithi137.stderr: -95> 2024-03-04T18:05:33.532+0000 7f76297fa640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.654 INFO:tasks.workunit.client.0.smithi137.stderr: -94> 2024-03-04T18:05:33.532+0000 7f7746951640 10 monclient: get_auth_request con 0x7f76100754c0 auth_method 0 2024-03-04T18:05:33.654 INFO:tasks.workunit.client.0.smithi137.stderr: -93> 2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:33.654 INFO:tasks.workunit.client.0.smithi137.stderr: -92> 2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:05:33.654 INFO:tasks.workunit.client.0.smithi137.stderr: -91> 2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:33.654 INFO:tasks.workunit.client.0.smithi137.stderr: -90> 2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:33.654 INFO:tasks.workunit.client.0.smithi137.stderr: -89> 2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:33.654 INFO:tasks.workunit.client.0.smithi137.stderr: -88> 2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:05:33.654 INFO:tasks.workunit.client.0.smithi137.stderr: -87> 2024-03-04T18:05:33.548+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.654 INFO:tasks.workunit.client.0.smithi137.stderr: -86> 2024-03-04T18:05:33.548+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -85> 2024-03-04T18:05:33.548+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -84> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -83> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -82> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -81> 2024-03-04T18:05:33.552+0000 7f774963a980 20 zone default found 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -80> 2024-03-04T18:05:33.552+0000 7f774963a980 4 Realm: () 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -79> 2024-03-04T18:05:33.552+0000 7f774963a980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -78> 2024-03-04T18:05:33.552+0000 7f774963a980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -77> 2024-03-04T18:05:33.552+0000 7f774963a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -76> 2024-03-04T18:05:33.552+0000 7f774963a980 20 zonegroup default 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -75> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -74> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -73> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -72> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -71> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.655 INFO:tasks.workunit.client.0.smithi137.stderr: -70> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.656 INFO:tasks.workunit.client.0.smithi137.stderr: -69> 2024-03-04T18:05:33.552+0000 7f774963a980 20 started sync module instance, tier type = 2024-03-04T18:05:33.656 INFO:tasks.workunit.client.0.smithi137.stderr: -68> 2024-03-04T18:05:33.552+0000 7f774963a980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:05:33.656 INFO:tasks.workunit.client.0.smithi137.stderr: -67> 2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=0 2024-03-04T18:05:33.656 INFO:tasks.workunit.client.0.smithi137.stderr: -66> 2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=1 2024-03-04T18:05:33.656 INFO:tasks.workunit.client.0.smithi137.stderr: -65> 2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=2 2024-03-04T18:05:33.656 INFO:tasks.workunit.client.0.smithi137.stderr: -64> 2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=3 2024-03-04T18:05:33.656 INFO:tasks.workunit.client.0.smithi137.stderr: -63> 2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=4 2024-03-04T18:05:33.656 INFO:tasks.workunit.client.0.smithi137.stderr: -62> 2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=5 2024-03-04T18:05:33.656 INFO:tasks.workunit.client.0.smithi137.stderr: -61> 2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=6 2024-03-04T18:05:33.656 INFO:tasks.workunit.client.0.smithi137.stderr: -60> 2024-03-04T18:05:33.580+0000 7f774963a980 20 add_watcher() i=7 2024-03-04T18:05:33.656 INFO:tasks.workunit.client.0.smithi137.stderr: -59> 2024-03-04T18:05:33.580+0000 7f774963a980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:33.656 INFO:tasks.workunit.client.0.smithi137.stderr: -58> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding auth protocol: cephx 2024-03-04T18:05:33.656 INFO:tasks.workunit.client.0.smithi137.stderr: -57> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding auth protocol: cephx 2024-03-04T18:05:33.657 INFO:tasks.workunit.client.0.smithi137.stderr: -56> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding auth protocol: cephx 2024-03-04T18:05:33.657 INFO:tasks.workunit.client.0.smithi137.stderr: -55> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: secure 2024-03-04T18:05:33.657 INFO:tasks.workunit.client.0.smithi137.stderr: -54> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: crc 2024-03-04T18:05:33.657 INFO:tasks.workunit.client.0.smithi137.stderr: -53> 2024-03-04T18:05:33.584+0000 7f75fe7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:33.657 INFO:tasks.workunit.client.0.smithi137.stderr: -52> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: secure 2024-03-04T18:05:33.657 INFO:tasks.workunit.client.0.smithi137.stderr: -51> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: crc 2024-03-04T18:05:33.657 INFO:tasks.workunit.client.0.smithi137.stderr: -50> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: secure 2024-03-04T18:05:33.657 INFO:tasks.workunit.client.0.smithi137.stderr: -49> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: crc 2024-03-04T18:05:33.657 INFO:tasks.workunit.client.0.smithi137.stderr: -48> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: crc 2024-03-04T18:05:33.657 INFO:tasks.workunit.client.0.smithi137.stderr: -47> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: secure 2024-03-04T18:05:33.657 INFO:tasks.workunit.client.0.smithi137.stderr: -46> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: crc 2024-03-04T18:05:33.657 INFO:tasks.workunit.client.0.smithi137.stderr: -45> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: secure 2024-03-04T18:05:33.657 INFO:tasks.workunit.client.0.smithi137.stderr: -44> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: crc 2024-03-04T18:05:33.657 INFO:tasks.workunit.client.0.smithi137.stderr: -43> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: secure 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -42> 2024-03-04T18:05:33.584+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -41> 2024-03-04T18:05:33.584+0000 7f774963a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -40> 2024-03-04T18:05:33.584+0000 7f774963a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -39> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command cache list hook 0x55871b0c9460 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -38> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command cache inspect hook 0x55871b0c9460 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -37> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command cache erase hook 0x55871b0c9460 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -36> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command cache zap hook 0x55871b0c9460 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -35> 2024-03-04T18:05:33.584+0000 7f774963a980 5 note: GC not initialized 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -34> 2024-03-04T18:05:33.584+0000 7f75fd7e2640 20 reqs_thread_entry: start 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -33> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command sync trace show hook 0x7f77380043b0 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -32> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command sync trace history hook 0x7f77380043b0 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -31> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command sync trace active hook 0x7f77380043b0 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -30> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command sync trace active_short hook 0x7f77380043b0 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -29> 2024-03-04T18:05:33.584+0000 7f774963a980 20 init_complete bucket index max shards: 1 2024-03-04T18:05:33.658 INFO:tasks.workunit.client.0.smithi137.stderr: -28> 2024-03-04T18:05:33.584+0000 7f774963a980 20 Filter name: none 2024-03-04T18:05:33.659 INFO:tasks.workunit.client.0.smithi137.stderr: -27> 2024-03-04T18:05:33.584+0000 7f75f67fc640 20 reqs_thread_entry: start 2024-03-04T18:05:33.659 INFO:tasks.workunit.client.0.smithi137.stderr: -26> 2024-03-04T18:05:33.588+0000 7f774963a980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-03-04T18:05:33.659 INFO:tasks.workunit.client.0.smithi137.stderr: -25> 2024-03-04T18:05:33.588+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.659 INFO:tasks.workunit.client.0.smithi137.stderr: -24> 2024-03-04T18:05:33.588+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:33.659 INFO:tasks.workunit.client.0.smithi137.stderr: -23> 2024-03-04T18:05:33.588+0000 7f774963a980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-03-04T18:05:33.659 INFO:tasks.workunit.client.0.smithi137.stderr: -22> 2024-03-04T18:05:33.588+0000 7f774963a980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-03-04T18:05:33.659 INFO:tasks.workunit.client.0.smithi137.stderr: -21> 2024-03-04T18:05:33.588+0000 7f774963a980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 : miss 2024-03-04T18:05:33.659 INFO:tasks.workunit.client.0.smithi137.stderr: -20> 2024-03-04T18:05:33.588+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.659 INFO:tasks.workunit.client.0.smithi137.stderr: -19> 2024-03-04T18:05:33.588+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:05:33.659 INFO:tasks.workunit.client.0.smithi137.stderr: -18> 2024-03-04T18:05:33.588+0000 7f774963a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 info.flags=0x17 2024-03-04T18:05:33.659 INFO:tasks.workunit.client.0.smithi137.stderr: -17> 2024-03-04T18:05:33.588+0000 7f774963a980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 to cache LRU end 2024-03-04T18:05:33.659 INFO:tasks.workunit.client.0.smithi137.stderr: -16> 2024-03-04T18:05:33.588+0000 7f774963a980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:05:33.659 INFO:tasks.workunit.client.0.smithi137.stderr: -15> 2024-03-04T18:05:33.588+0000 7f774963a980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:33.659 INFO:tasks.workunit.client.0.smithi137.stderr: -14> 2024-03-04T18:05:33.588+0000 7f774963a980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 2024-03-04T18:05:33.660 INFO:tasks.workunit.client.0.smithi137.stderr: -13> 2024-03-04T18:05:33.596+0000 7f774963a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 info.flags=0x17 2024-03-04T18:05:33.660 INFO:tasks.workunit.client.0.smithi137.stderr: -12> 2024-03-04T18:05:33.596+0000 7f774963a980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 to cache LRU end 2024-03-04T18:05:33.660 INFO:tasks.workunit.client.0.smithi137.stderr: -11> 2024-03-04T18:05:33.596+0000 7f774963a980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:33.660 INFO:tasks.workunit.client.0.smithi137.stderr: -10> 2024-03-04T18:05:33.596+0000 7f774963a980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29, ofs0, ns] 2024-03-04T18:05:33.660 INFO:tasks.workunit.client.0.smithi137.stderr: -9> 2024-03-04T18:05:33.596+0000 7f762a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627796 cookie 94038762984912 notifier 6933 bl.length()=945 2024-03-04T18:05:33.660 INFO:tasks.workunit.client.0.smithi137.stderr: -8> 2024-03-04T18:05:33.596+0000 7f762a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 info.flags=0x17 2024-03-04T18:05:33.660 INFO:tasks.workunit.client.0.smithi137.stderr: -7> 2024-03-04T18:05:33.596+0000 7f762a7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 to cache LRU end 2024-03-04T18:05:33.660 INFO:tasks.workunit.client.0.smithi137.stderr: -6> 2024-03-04T18:05:33.596+0000 7f762a7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:33.660 INFO:tasks.workunit.client.0.smithi137.stderr: -5> 2024-03-04T18:05:33.600+0000 7f774963a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): orig_sources= new_sources= 2024-03-04T18:05:33.660 INFO:tasks.workunit.client.0.smithi137.stderr: -4> 2024-03-04T18:05:33.600+0000 7f774963a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): potential sources added=[] removed=[] 2024-03-04T18:05:33.660 INFO:tasks.workunit.client.0.smithi137.stderr: -3> 2024-03-04T18:05:33.600+0000 7f774963a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): orig_dests= new_dests= 2024-03-04T18:05:33.660 INFO:tasks.workunit.client.0.smithi137.stderr: -2> 2024-03-04T18:05:33.600+0000 7f774963a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): potential dests added=[] removed=[] 2024-03-04T18:05:33.660 INFO:tasks.workunit.client.0.smithi137.stderr: -1> 2024-03-04T18:05:33.604+0000 7f774963a980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f774963a980 time 2024-03-04T18:05:33.604827+0000 2024-03-04T18:05:33.660 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f774cf0fd7b] 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f774cf0ff2f] 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558719a10281] 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x558719a1cd5d] 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: 5: main() 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f774bd7ad90] 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: 7: __libc_start_main() 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: 8: _start() 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: 0> 2024-03-04T18:05:33.604+0000 7f774963a980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7f774963a980 thread_name:radosgw-admin 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:33.661 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f774bd93520] 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f774cf0fdd5] 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f774cf0ff2f] 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558719a10281] 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x558719a1cd5d] 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: 9: main() 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f774bd7ad90] 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: 11: __libc_start_main() 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: 12: _start() 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr:--- logging levels --- 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 none 2024-03-04T18:05:33.662 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 lockdep 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 context 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 crush 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_balancer 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_locker 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log_expire 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_migrator 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 buffer 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 timer 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 filer 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 striper 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 objecter 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rados 2024-03-04T18:05:33.663 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_mirror 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_replay 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_pwl 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 journaler 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objectcacher 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 client 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 osd 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 optracker 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objclass 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 filestore 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 journal 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 ms 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mon 2024-03-04T18:05:33.664 INFO:tasks.workunit.client.0.smithi137.stderr: 0/10 monc 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 paxos 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 tp 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 auth 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 crypto 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 finisher 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 reserver 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 heartbeatmap 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 perfcounter 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 20/20 rgw 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_sync 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_datacache 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_access 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_flight 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:05:33.665 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 javaclient 2024-03-04T18:05:33.666 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 asok 2024-03-04T18:05:33.666 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 throttle 2024-03-04T18:05:33.666 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 refs 2024-03-04T18:05:33.666 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 compressor 2024-03-04T18:05:33.666 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluestore 2024-03-04T18:05:33.666 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluefs 2024-03-04T18:05:33.666 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 bdev 2024-03-04T18:05:33.666 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 kstore 2024-03-04T18:05:33.666 INFO:tasks.workunit.client.0.smithi137.stderr: 4/ 5 rocksdb 2024-03-04T18:05:33.666 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 fuse 2024-03-04T18:05:33.666 INFO:tasks.workunit.client.0.smithi137.stderr: 2/ 5 mgr 2024-03-04T18:05:33.666 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mgrc 2024-03-04T18:05:33.666 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 dpdk 2024-03-04T18:05:33.666 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 eventtrace 2024-03-04T18:05:33.667 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 prioritycache 2024-03-04T18:05:33.667 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 test 2024-03-04T18:05:33.667 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:05:33.667 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephsqlite 2024-03-04T18:05:33.667 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:05:33.667 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore 2024-03-04T18:05:33.667 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_onode 2024-03-04T18:05:33.667 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_odata 2024-03-04T18:05:33.667 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_omap 2024-03-04T18:05:33.667 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_tm 2024-03-04T18:05:33.667 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_t 2024-03-04T18:05:33.667 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:05:33.667 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_epm 2024-03-04T18:05:33.667 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_lba 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cache 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_journal 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_device 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_backref 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 alienstore 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mclock 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cyanstore 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 ceph_exporter 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 memstore 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 trace 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr: -2/-2 (syslog threshold) 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr: 99/99 (stderr threshold) 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:05:33.668 INFO:tasks.workunit.client.0.smithi137.stderr: 7f75f67fc640 / http_manager 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: 7f75fd7e2640 / http_manager 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: 7f75fe7e4640 / rgw_dt_lg_renew 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: 7f76297fa640 / ms_dispatch 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: 7f762a7fc640 / io_context_pool 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: 7f762affd640 / io_context_pool 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: 7f7631602640 / ms_dispatch 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: 7f7633e07640 / ms_dispatch 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: 7f763560a640 / io_context_pool 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: 7f7746150640 / io_context_pool 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: 7f7746951640 / msgr-worker-2 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: 7f7747152640 / msgr-worker-1 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: 7f7747953640 / msgr-worker-0 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: 7f774963a980 / radosgw-admin 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: max_recent 500 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: max_new 1000 2024-03-04T18:05:33.669 INFO:tasks.workunit.client.0.smithi137.stderr: log_file /var/log/ceph/ceph-client.admin.101842.log 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr:--- end dump of recent events --- 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr:--- begin dump of recent events --- 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr: -499> 2024-03-04T18:05:33.292+0000 7f7747152640 1 monclient: found mon.noname-c 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr: -498> 2024-03-04T18:05:33.292+0000 7f7747152640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.137:3301/0 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr: -497> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr: -496> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr: -495> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: dump: 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.670 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.671 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.671 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.671 INFO:tasks.workunit.client.0.smithi137.stderr: -494> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: _finish_auth 0 2024-03-04T18:05:33.671 INFO:tasks.workunit.client.0.smithi137.stderr: -493> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: _check_auth_tickets 2024-03-04T18:05:33.671 INFO:tasks.workunit.client.0.smithi137.stderr: -492> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:33.671 INFO:tasks.workunit.client.0.smithi137.stderr: -491> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.671 INFO:tasks.workunit.client.0.smithi137.stderr: -490> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-03-04T18:05:33.671 INFO:tasks.workunit.client.0.smithi137.stderr: -489> 2024-03-04T18:05:33.292+0000 7f763560a640 10 monclient: dump: 2024-03-04T18:05:33.671 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.671 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.671 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.671 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.671 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.671 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr: -488> 2024-03-04T18:05:33.292+0000 7f774963a980 5 monclient: authenticate success, global_id 6542 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr: -487> 2024-03-04T18:05:33.292+0000 7f774963a980 10 monclient: get_monmap_and_config success 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr: -486> 2024-03-04T18:05:33.292+0000 7f774963a980 4 set_mon_vals no callback set 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr: -485> 2024-03-04T18:05:33.292+0000 7f774963a980 10 monclient: shutdown 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr: -484> 2024-03-04T18:05:33.292+0000 7f774963a980 5 asok(0x55871ae20130) unregister_commands rotate-key 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr: -483> 2024-03-04T18:05:33.292+0000 7f774963a980 10 monclient: build_initial_monmap 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr: -482> 2024-03-04T18:05:33.292+0000 7f774963a980 1 build_initial for_mkfs: 0 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr: -481> 2024-03-04T18:05:33.292+0000 7f774963a980 10 monclient: monmap: 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:05:33.672 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:05:33.673 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:05:33.673 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:05:33.673 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.673 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:05:33.673 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:05:33.673 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:05:33.673 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.673 INFO:tasks.workunit.client.0.smithi137.stderr: -480> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding auth protocol: cephx 2024-03-04T18:05:33.673 INFO:tasks.workunit.client.0.smithi137.stderr: -479> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding auth protocol: cephx 2024-03-04T18:05:33.673 INFO:tasks.workunit.client.0.smithi137.stderr: -478> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding auth protocol: cephx 2024-03-04T18:05:33.673 INFO:tasks.workunit.client.0.smithi137.stderr: -477> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: secure 2024-03-04T18:05:33.673 INFO:tasks.workunit.client.0.smithi137.stderr: -476> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: crc 2024-03-04T18:05:33.673 INFO:tasks.workunit.client.0.smithi137.stderr: -475> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: secure 2024-03-04T18:05:33.673 INFO:tasks.workunit.client.0.smithi137.stderr: -474> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: crc 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -473> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: secure 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -472> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: crc 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -471> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: crc 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -470> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: secure 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -469> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: crc 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -468> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: secure 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -467> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: crc 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -466> 2024-03-04T18:05:33.292+0000 7f774963a980 5 AuthRegistry(0x55871b037310) adding con mode: secure 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -465> 2024-03-04T18:05:33.296+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -464> 2024-03-04T18:05:33.296+0000 7f774963a980 1 librados: starting msgr at 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -463> 2024-03-04T18:05:33.296+0000 7f774963a980 1 librados: starting objecter 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -462> 2024-03-04T18:05:33.296+0000 7f774963a980 5 asok(0x55871ae20130) register_command objecter_requests hook 0x55871adbe4c0 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -461> 2024-03-04T18:05:33.296+0000 7f774963a980 1 librados: setting wanted keys 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -460> 2024-03-04T18:05:33.296+0000 7f774963a980 1 librados: calling monclient init 2024-03-04T18:05:33.674 INFO:tasks.workunit.client.0.smithi137.stderr: -459> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient: init 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -458> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding auth protocol: cephx 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -457> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding auth protocol: cephx 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -456> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding auth protocol: cephx 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -455> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: secure 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -454> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: crc 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -453> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: secure 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -452> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: crc 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -451> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: secure 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -450> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: crc 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -449> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: crc 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -448> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: secure 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -447> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: crc 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -446> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: secure 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -445> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: crc 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -444> 2024-03-04T18:05:33.296+0000 7f774963a980 5 AuthRegistry(0x55871b007580) adding con mode: secure 2024-03-04T18:05:33.675 INFO:tasks.workunit.client.0.smithi137.stderr: -443> 2024-03-04T18:05:33.296+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.676 INFO:tasks.workunit.client.0.smithi137.stderr: -442> 2024-03-04T18:05:33.296+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.676 INFO:tasks.workunit.client.0.smithi137.stderr: -441> 2024-03-04T18:05:33.296+0000 7f774963a980 5 asok(0x55871ae20130) register_command rotate-key hook 0x55871b0076c8 2024-03-04T18:05:33.676 INFO:tasks.workunit.client.0.smithi137.stderr: -440> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:33.676 INFO:tasks.workunit.client.0.smithi137.stderr: -439> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient: _add_conns ranks=[0,1,2] 2024-03-04T18:05:33.676 INFO:tasks.workunit.client.0.smithi137.stderr: -438> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): picked mon.noname-a con 0x55871aeb6370 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:05:33.676 INFO:tasks.workunit.client.0.smithi137.stderr: -437> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): picked mon.noname-b con 0x55871aec1280 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:05:33.676 INFO:tasks.workunit.client.0.smithi137.stderr: -436> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): picked mon.noname-c con 0x55871af4f400 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:05:33.676 INFO:tasks.workunit.client.0.smithi137.stderr: -435> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.676 INFO:tasks.workunit.client.0.smithi137.stderr: -434> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.676 INFO:tasks.workunit.client.0.smithi137.stderr: -433> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.676 INFO:tasks.workunit.client.0.smithi137.stderr: -432> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:33.676 INFO:tasks.workunit.client.0.smithi137.stderr: -431> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient(hunting): authenticate will time out at 2566.412932s 2024-03-04T18:05:33.676 INFO:tasks.workunit.client.0.smithi137.stderr: -430> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): get_auth_request con 0x55871aeb6370 auth_method 0 2024-03-04T18:05:33.676 INFO:tasks.workunit.client.0.smithi137.stderr: -429> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.677 INFO:tasks.workunit.client.0.smithi137.stderr: -428> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.677 INFO:tasks.workunit.client.0.smithi137.stderr: -427> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.677 INFO:tasks.workunit.client.0.smithi137.stderr: -426> 2024-03-04T18:05:33.296+0000 7f7747953640 10 monclient(hunting): get_auth_request con 0x55871af4f400 auth_method 0 2024-03-04T18:05:33.677 INFO:tasks.workunit.client.0.smithi137.stderr: -425> 2024-03-04T18:05:33.296+0000 7f7747953640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.677 INFO:tasks.workunit.client.0.smithi137.stderr: -424> 2024-03-04T18:05:33.296+0000 7f7747953640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.677 INFO:tasks.workunit.client.0.smithi137.stderr: -423> 2024-03-04T18:05:33.296+0000 7f7747953640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.677 INFO:tasks.workunit.client.0.smithi137.stderr: -422> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.677 INFO:tasks.workunit.client.0.smithi137.stderr: -421> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.677 INFO:tasks.workunit.client.0.smithi137.stderr: -420> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.677 INFO:tasks.workunit.client.0.smithi137.stderr: -419> 2024-03-04T18:05:33.296+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.677 INFO:tasks.workunit.client.0.smithi137.stderr: -418> 2024-03-04T18:05:33.296+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.677 INFO:tasks.workunit.client.0.smithi137.stderr: -417> 2024-03-04T18:05:33.296+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.677 INFO:tasks.workunit.client.0.smithi137.stderr: -416> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient(hunting): handle_auth_done global_id 6924 payload 824 2024-03-04T18:05:33.677 INFO:tasks.workunit.client.0.smithi137.stderr: -415> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient: _finish_hunting 0 2024-03-04T18:05:33.678 INFO:tasks.workunit.client.0.smithi137.stderr: -414> 2024-03-04T18:05:33.296+0000 7f7747152640 1 monclient: found mon.noname-a 2024-03-04T18:05:33.678 INFO:tasks.workunit.client.0.smithi137.stderr: -413> 2024-03-04T18:05:33.296+0000 7f7747152640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.678 INFO:tasks.workunit.client.0.smithi137.stderr: -412> 2024-03-04T18:05:33.296+0000 7f7746951640 10 monclient: get_auth_request con 0x55871aec1280 auth_method 0 2024-03-04T18:05:33.678 INFO:tasks.workunit.client.0.smithi137.stderr: -411> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.678 INFO:tasks.workunit.client.0.smithi137.stderr: -410> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:05:33.678 INFO:tasks.workunit.client.0.smithi137.stderr: -409> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: dump: 2024-03-04T18:05:33.678 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.678 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.678 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.678 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.678 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.678 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.678 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.678 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr: -408> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: _finish_auth 0 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr: -407> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: _check_auth_tickets 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr: -406> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr: -405> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr: -404> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr: -403> 2024-03-04T18:05:33.296+0000 7f7633e07640 10 monclient: dump: 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.679 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr: -402> 2024-03-04T18:05:33.296+0000 7f774963a980 5 monclient: authenticate success, global_id 6924 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr: -401> 2024-03-04T18:05:33.296+0000 7f763560a640 4 set_mon_vals no callback set 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr: -400> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient: _renew_subs 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr: -399> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr: -398> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient: _renew_subs 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr: -397> 2024-03-04T18:05:33.296+0000 7f774963a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr: -396> 2024-03-04T18:05:33.296+0000 7f774963a980 1 librados: init done 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr: -395> 2024-03-04T18:05:33.296+0000 7f7633e07640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr: -394> 2024-03-04T18:05:33.296+0000 7f7633e07640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr: -393> 2024-03-04T18:05:33.296+0000 7f7633e07640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr: -392> 2024-03-04T18:05:33.300+0000 7f7746951640 10 monclient: get_auth_request con 0x7f762c065940 auth_method 0 2024-03-04T18:05:33.680 INFO:tasks.workunit.client.0.smithi137.stderr: -391> 2024-03-04T18:05:33.300+0000 7f7747953640 10 monclient: get_auth_request con 0x55871b034c70 auth_method 0 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr: -390> 2024-03-04T18:05:33.300+0000 7f7747152640 10 monclient: get_auth_request con 0x55871b052860 auth_method 0 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr: -389> 2024-03-04T18:05:33.300+0000 7f7746951640 10 monclient: get_auth_request con 0x55871b056a50 auth_method 0 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr: -388> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: build_initial_monmap 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr: -387> 2024-03-04T18:05:33.304+0000 7f774963a980 1 build_initial for_mkfs: 0 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr: -386> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: monmap: 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:05:33.681 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -385> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding auth protocol: cephx 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -384> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding auth protocol: cephx 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -383> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding auth protocol: cephx 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -382> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: secure 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -381> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: crc 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -380> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: secure 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -379> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: crc 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -378> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: secure 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -377> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: crc 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -376> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: crc 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -375> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: secure 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -374> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: crc 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -373> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: secure 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -372> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: crc 2024-03-04T18:05:33.682 INFO:tasks.workunit.client.0.smithi137.stderr: -371> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b05b4a0) adding con mode: secure 2024-03-04T18:05:33.683 INFO:tasks.workunit.client.0.smithi137.stderr: -370> 2024-03-04T18:05:33.304+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.683 INFO:tasks.workunit.client.0.smithi137.stderr: -369> 2024-03-04T18:05:33.304+0000 7f774963a980 5 asok(0x55871ae20130) register_command objecter_requests cmddesc objecter_requests hook 0x55871b060cb0 EEXIST 2024-03-04T18:05:33.683 INFO:tasks.workunit.client.0.smithi137.stderr: -368> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: init 2024-03-04T18:05:33.683 INFO:tasks.workunit.client.0.smithi137.stderr: -367> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding auth protocol: cephx 2024-03-04T18:05:33.683 INFO:tasks.workunit.client.0.smithi137.stderr: -366> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding auth protocol: cephx 2024-03-04T18:05:33.683 INFO:tasks.workunit.client.0.smithi137.stderr: -365> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding auth protocol: cephx 2024-03-04T18:05:33.683 INFO:tasks.workunit.client.0.smithi137.stderr: -364> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: secure 2024-03-04T18:05:33.683 INFO:tasks.workunit.client.0.smithi137.stderr: -363> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: crc 2024-03-04T18:05:33.683 INFO:tasks.workunit.client.0.smithi137.stderr: -362> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: secure 2024-03-04T18:05:33.683 INFO:tasks.workunit.client.0.smithi137.stderr: -361> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: crc 2024-03-04T18:05:33.683 INFO:tasks.workunit.client.0.smithi137.stderr: -360> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: secure 2024-03-04T18:05:33.683 INFO:tasks.workunit.client.0.smithi137.stderr: -359> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: crc 2024-03-04T18:05:33.683 INFO:tasks.workunit.client.0.smithi137.stderr: -358> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: crc 2024-03-04T18:05:33.683 INFO:tasks.workunit.client.0.smithi137.stderr: -357> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: secure 2024-03-04T18:05:33.684 INFO:tasks.workunit.client.0.smithi137.stderr: -356> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: crc 2024-03-04T18:05:33.684 INFO:tasks.workunit.client.0.smithi137.stderr: -355> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: secure 2024-03-04T18:05:33.684 INFO:tasks.workunit.client.0.smithi137.stderr: -354> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: crc 2024-03-04T18:05:33.684 INFO:tasks.workunit.client.0.smithi137.stderr: -353> 2024-03-04T18:05:33.304+0000 7f774963a980 5 AuthRegistry(0x55871b059c68) adding con mode: secure 2024-03-04T18:05:33.684 INFO:tasks.workunit.client.0.smithi137.stderr: -352> 2024-03-04T18:05:33.304+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.684 INFO:tasks.workunit.client.0.smithi137.stderr: -351> 2024-03-04T18:05:33.304+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.684 INFO:tasks.workunit.client.0.smithi137.stderr: -350> 2024-03-04T18:05:33.304+0000 7f774963a980 5 asok(0x55871ae20130) register_command rotate-key cmddesc rotate-key hook 0x55871b059db0 EEXIST 2024-03-04T18:05:33.684 INFO:tasks.workunit.client.0.smithi137.stderr: -349> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:33.684 INFO:tasks.workunit.client.0.smithi137.stderr: -348> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: _add_conns ranks=[1,2,0] 2024-03-04T18:05:33.684 INFO:tasks.workunit.client.0.smithi137.stderr: -347> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): picked mon.noname-b con 0x55871b085910 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:05:33.684 INFO:tasks.workunit.client.0.smithi137.stderr: -346> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): picked mon.noname-c con 0x55871b085cf0 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:05:33.684 INFO:tasks.workunit.client.0.smithi137.stderr: -345> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): picked mon.noname-a con 0x55871b080eb0 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:05:33.684 INFO:tasks.workunit.client.0.smithi137.stderr: -344> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.684 INFO:tasks.workunit.client.0.smithi137.stderr: -343> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -342> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -341> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -340> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient(hunting): authenticate will time out at 2566.421309s 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -339> 2024-03-04T18:05:33.304+0000 7f7746951640 10 monclient(hunting): get_auth_request con 0x55871b085cf0 auth_method 0 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -338> 2024-03-04T18:05:33.304+0000 7f7746951640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -337> 2024-03-04T18:05:33.304+0000 7f7746951640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -336> 2024-03-04T18:05:33.304+0000 7f7746951640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -335> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): get_auth_request con 0x55871b080eb0 auth_method 0 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -334> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -333> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -332> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -331> 2024-03-04T18:05:33.304+0000 7f7746951640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -330> 2024-03-04T18:05:33.304+0000 7f7746951640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -329> 2024-03-04T18:05:33.304+0000 7f7746951640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.685 INFO:tasks.workunit.client.0.smithi137.stderr: -328> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr: -327> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr: -326> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr: -325> 2024-03-04T18:05:33.304+0000 7f7747152640 10 monclient(hunting): get_auth_request con 0x55871b085910 auth_method 0 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr: -324> 2024-03-04T18:05:33.304+0000 7f7747152640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr: -323> 2024-03-04T18:05:33.304+0000 7f7747152640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr: -322> 2024-03-04T18:05:33.304+0000 7f7747152640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr: -321> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient(hunting): handle_auth_done global_id 6927 payload 824 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr: -320> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient: _finish_hunting 0 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr: -319> 2024-03-04T18:05:33.304+0000 7f7747953640 1 monclient: found mon.noname-a 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr: -318> 2024-03-04T18:05:33.304+0000 7f7747953640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr: -317> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr: -316> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr: -315> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: dump: 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.686 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.687 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.687 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.687 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.687 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.687 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.687 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.687 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.687 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.687 INFO:tasks.workunit.client.0.smithi137.stderr: -314> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: _finish_auth 0 2024-03-04T18:05:33.687 INFO:tasks.workunit.client.0.smithi137.stderr: -313> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: _check_auth_tickets 2024-03-04T18:05:33.687 INFO:tasks.workunit.client.0.smithi137.stderr: -312> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:33.687 INFO:tasks.workunit.client.0.smithi137.stderr: -311> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.687 INFO:tasks.workunit.client.0.smithi137.stderr: -310> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:05:33.687 INFO:tasks.workunit.client.0.smithi137.stderr: -309> 2024-03-04T18:05:33.304+0000 7f7631602640 10 monclient: dump: 2024-03-04T18:05:33.688 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.688 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.688 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.688 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.688 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.688 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.688 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.688 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.688 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.688 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.688 INFO:tasks.workunit.client.0.smithi137.stderr: -308> 2024-03-04T18:05:33.304+0000 7f774963a980 5 monclient: authenticate success, global_id 6927 2024-03-04T18:05:33.688 INFO:tasks.workunit.client.0.smithi137.stderr: -307> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: _renew_subs 2024-03-04T18:05:33.688 INFO:tasks.workunit.client.0.smithi137.stderr: -306> 2024-03-04T18:05:33.304+0000 7f7746150640 4 set_mon_vals no callback set 2024-03-04T18:05:33.688 INFO:tasks.workunit.client.0.smithi137.stderr: -305> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr: -304> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: _renew_subs 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr: -303> 2024-03-04T18:05:33.304+0000 7f774963a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr: -302> 2024-03-04T18:05:33.308+0000 7f7631602640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr: -301> 2024-03-04T18:05:33.308+0000 7f7631602640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr: -300> 2024-03-04T18:05:33.308+0000 7f7631602640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr: -299> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: get_monmap_and_config 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr: -298> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: build_initial_monmap 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr: -297> 2024-03-04T18:05:33.308+0000 7f774963a980 1 build_initial for_mkfs: 0 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr: -296> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: monmap: 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:05:33.689 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr: -295> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding auth protocol: cephx 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr: -294> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding auth protocol: cephx 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr: -293> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding auth protocol: cephx 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr: -292> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr: -291> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr: -290> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr: -289> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr: -288> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr: -287> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr: -286> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.690 INFO:tasks.workunit.client.0.smithi137.stderr: -285> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -284> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -283> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -282> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -281> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -280> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient: get_auth_request con 0x7f7620065870 auth_method 0 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -279> 2024-03-04T18:05:33.308+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -278> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: init 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -277> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding auth protocol: cephx 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -276> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding auth protocol: cephx 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -275> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding auth protocol: cephx 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -274> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: secure 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -273> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: crc 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -272> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: secure 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -271> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: crc 2024-03-04T18:05:33.691 INFO:tasks.workunit.client.0.smithi137.stderr: -270> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: secure 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -269> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: crc 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -268> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: crc 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -267> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: secure 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -266> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: crc 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -265> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: secure 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -264> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: crc 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -263> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x7ffe8148f6e0) adding con mode: secure 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -262> 2024-03-04T18:05:33.308+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -261> 2024-03-04T18:05:33.308+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -260> 2024-03-04T18:05:33.308+0000 7f774963a980 5 asok(0x55871ae20130) register_command rotate-key cmddesc rotate-key hook 0x7ffe8148f828 EEXIST 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -259> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -258> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: _add_conns ranks=[0,1,2] 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -257> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): picked mon.noname-a con 0x55871b0d7b60 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -256> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): picked mon.noname-b con 0x55871b0d7f40 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:05:33.692 INFO:tasks.workunit.client.0.smithi137.stderr: -255> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): picked mon.noname-c con 0x55871b0dd360 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -254> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -253> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -252> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -251> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -250> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): authenticate will time out at 2566.425414s 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -249> 2024-03-04T18:05:33.308+0000 7f7747953640 10 monclient(hunting): get_auth_request con 0x55871b0dd360 auth_method 0 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -248> 2024-03-04T18:05:33.308+0000 7f7747953640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -247> 2024-03-04T18:05:33.308+0000 7f7747953640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -246> 2024-03-04T18:05:33.308+0000 7f7747953640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -245> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): get_auth_request con 0x55871b0d7b60 auth_method 0 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -244> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -243> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -242> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -241> 2024-03-04T18:05:33.308+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -240> 2024-03-04T18:05:33.308+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.693 INFO:tasks.workunit.client.0.smithi137.stderr: -239> 2024-03-04T18:05:33.308+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.694 INFO:tasks.workunit.client.0.smithi137.stderr: -238> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.694 INFO:tasks.workunit.client.0.smithi137.stderr: -237> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.694 INFO:tasks.workunit.client.0.smithi137.stderr: -236> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.694 INFO:tasks.workunit.client.0.smithi137.stderr: -235> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient(hunting): handle_auth_done global_id 6930 payload 274 2024-03-04T18:05:33.694 INFO:tasks.workunit.client.0.smithi137.stderr: -234> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient: _finish_hunting 0 2024-03-04T18:05:33.694 INFO:tasks.workunit.client.0.smithi137.stderr: -233> 2024-03-04T18:05:33.308+0000 7f7747152640 1 monclient: found mon.noname-a 2024-03-04T18:05:33.694 INFO:tasks.workunit.client.0.smithi137.stderr: -232> 2024-03-04T18:05:33.308+0000 7f7747152640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.694 INFO:tasks.workunit.client.0.smithi137.stderr: -231> 2024-03-04T18:05:33.308+0000 7f7746951640 10 monclient: get_auth_request con 0x55871b0d7f40 auth_method 0 2024-03-04T18:05:33.694 INFO:tasks.workunit.client.0.smithi137.stderr: -230> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.694 INFO:tasks.workunit.client.0.smithi137.stderr: -229> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:05:33.694 INFO:tasks.workunit.client.0.smithi137.stderr: -228> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: dump: 2024-03-04T18:05:33.694 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.694 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.694 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr: -227> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: _finish_auth 0 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr: -226> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: _check_auth_tickets 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr: -225> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr: -224> 2024-03-04T18:05:33.308+0000 7f774963a980 5 monclient: authenticate success, global_id 6930 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr: -223> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: get_monmap_and_config success 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr: -222> 2024-03-04T18:05:33.308+0000 7f774963a980 4 set_mon_vals no callback set 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr: -221> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.695 INFO:tasks.workunit.client.0.smithi137.stderr: -220> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:05:33.696 INFO:tasks.workunit.client.0.smithi137.stderr: -219> 2024-03-04T18:05:33.308+0000 7f762affd640 10 monclient: dump: 2024-03-04T18:05:33.696 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.696 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.696 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.696 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.696 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.696 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.696 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.696 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.696 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.696 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.696 INFO:tasks.workunit.client.0.smithi137.stderr: -218> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: shutdown 2024-03-04T18:05:33.696 INFO:tasks.workunit.client.0.smithi137.stderr: -217> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: build_initial_monmap 2024-03-04T18:05:33.696 INFO:tasks.workunit.client.0.smithi137.stderr: -216> 2024-03-04T18:05:33.308+0000 7f774963a980 1 build_initial for_mkfs: 0 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr: -215> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: monmap: 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 0 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 0.000000 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr:created 0.000000 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 0 (unknown) 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.noname-a 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.noname-b 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.noname-c 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr: -214> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding auth protocol: cephx 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr: -213> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding auth protocol: cephx 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr: -212> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding auth protocol: cephx 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr: -211> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.697 INFO:tasks.workunit.client.0.smithi137.stderr: -210> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -209> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -208> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -207> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -206> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -205> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -204> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -203> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -202> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -201> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: crc 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -200> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a3460) adding con mode: secure 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -199> 2024-03-04T18:05:33.308+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -198> 2024-03-04T18:05:33.308+0000 7f774963a980 1 librados: starting msgr at 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -197> 2024-03-04T18:05:33.308+0000 7f774963a980 1 librados: starting objecter 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -196> 2024-03-04T18:05:33.308+0000 7f774963a980 5 asok(0x55871ae20130) register_command objecter_requests cmddesc objecter_requests hook 0x55871b0dcd90 EEXIST 2024-03-04T18:05:33.698 INFO:tasks.workunit.client.0.smithi137.stderr: -195> 2024-03-04T18:05:33.308+0000 7f774963a980 1 librados: setting wanted keys 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -194> 2024-03-04T18:05:33.308+0000 7f774963a980 1 librados: calling monclient init 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -193> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: init 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -192> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding auth protocol: cephx 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -191> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding auth protocol: cephx 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -190> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding auth protocol: cephx 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -189> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: secure 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -188> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: crc 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -187> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: secure 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -186> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: crc 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -185> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: secure 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -184> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: crc 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -183> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: crc 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -182> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: secure 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -181> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: crc 2024-03-04T18:05:33.699 INFO:tasks.workunit.client.0.smithi137.stderr: -180> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: secure 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -179> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: crc 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -178> 2024-03-04T18:05:33.308+0000 7f774963a980 5 AuthRegistry(0x55871b0a2630) adding con mode: secure 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -177> 2024-03-04T18:05:33.308+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -176> 2024-03-04T18:05:33.308+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -175> 2024-03-04T18:05:33.308+0000 7f774963a980 5 asok(0x55871ae20130) register_command rotate-key cmddesc rotate-key hook 0x55871b0a2778 EEXIST 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -174> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: _reopen_session rank -1 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -173> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient: _add_conns ranks=[0,1,2] 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -172> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): picked mon.noname-a con 0x55871b0d7b60 addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -171> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): picked mon.noname-b con 0x55871b0dd360 addr [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -170> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): picked mon.noname-c con 0x55871b0dad40 addr [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -169> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -168> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -167> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): start opening mon connection 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -166> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): _renew_subs 2024-03-04T18:05:33.700 INFO:tasks.workunit.client.0.smithi137.stderr: -165> 2024-03-04T18:05:33.308+0000 7f774963a980 10 monclient(hunting): authenticate will time out at 2566.427960s 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -164> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): get_auth_request con 0x55871b0d7b60 auth_method 0 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -163> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -162> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -161> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -160> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -159> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -158> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -157> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient(hunting): get_auth_request con 0x55871b0dad40 auth_method 0 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -156> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -155> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -154> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -153> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -152> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -151> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-03-04T18:05:33.701 INFO:tasks.workunit.client.0.smithi137.stderr: -150> 2024-03-04T18:05:33.312+0000 7f7746951640 10 monclient(hunting): get_auth_request con 0x55871b0dd360 auth_method 0 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr: -149> 2024-03-04T18:05:33.312+0000 7f7746951640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr: -148> 2024-03-04T18:05:33.312+0000 7f7746951640 10 monclient(hunting): _init_auth method 2 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr: -147> 2024-03-04T18:05:33.312+0000 7f7746951640 10 monclient(hunting): _init_auth creating new auth 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr: -146> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient(hunting): handle_auth_done global_id 6933 payload 824 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr: -145> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient: _finish_hunting 0 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr: -144> 2024-03-04T18:05:33.312+0000 7f7747152640 1 monclient: found mon.noname-a 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr: -143> 2024-03-04T18:05:33.312+0000 7f7747152640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr: -142> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr: -141> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr: -140> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: dump: 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.702 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr: -139> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: _finish_auth 0 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr: -138> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: _check_auth_tickets 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr: -137> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: handle_config config(0 keys) v1 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr: -136> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: handle_monmap mon_map magic: 0 v1 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr: -135> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr: -134> 2024-03-04T18:05:33.312+0000 7f76297fa640 10 monclient: dump: 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr:epoch 1 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr:fsid 4e7d04e9-67dd-413e-a1a3-7566f727539e 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr:last_changed 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.703 INFO:tasks.workunit.client.0.smithi137.stderr:created 2024-03-04T17:39:35.298234+0000 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr:min_mon_release 19 (squid) 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr:election_strategy: 1 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr:0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr: -133> 2024-03-04T18:05:33.312+0000 7f774963a980 5 monclient: authenticate success, global_id 6933 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr: -132> 2024-03-04T18:05:33.312+0000 7f774963a980 10 monclient: _renew_subs 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr: -131> 2024-03-04T18:05:33.312+0000 7f762affd640 4 set_mon_vals no callback set 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr: -130> 2024-03-04T18:05:33.312+0000 7f774963a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr: -129> 2024-03-04T18:05:33.312+0000 7f774963a980 10 monclient: _renew_subs 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr: -128> 2024-03-04T18:05:33.312+0000 7f774963a980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr: -127> 2024-03-04T18:05:33.312+0000 7f774963a980 1 librados: init done 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr: -126> 2024-03-04T18:05:33.312+0000 7f774963a980 5 asok(0x55871ae20130) register_command cr dump hook 0x55871b0c9ee8 2024-03-04T18:05:33.704 INFO:tasks.workunit.client.0.smithi137.stderr: -125> 2024-03-04T18:05:33.312+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -124> 2024-03-04T18:05:33.312+0000 7f76297fa640 4 mgrc handle_mgr_map Got map version 6 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -123> 2024-03-04T18:05:33.312+0000 7f76297fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -122> 2024-03-04T18:05:33.312+0000 7f76297fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.173:6832/3535769218,v1:172.21.15.173:6833/3535769218] 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -121> 2024-03-04T18:05:33.312+0000 7f7746951640 10 monclient: get_auth_request con 0x7f7610065940 auth_method 0 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -120> 2024-03-04T18:05:33.312+0000 7f7747953640 10 monclient: get_auth_request con 0x55871b0f0e20 auth_method 0 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -119> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -118> 2024-03-04T18:05:33.316+0000 7f774963a980 20 realm 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -117> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -116> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -115> 2024-03-04T18:05:33.316+0000 7f774963a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -114> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -113> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -112> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -111> 2024-03-04T18:05:33.316+0000 7f7747152640 10 monclient: get_auth_request con 0x55871b0f5030 auth_method 0 2024-03-04T18:05:33.705 INFO:tasks.workunit.client.0.smithi137.stderr: -110> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -109> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -108> 2024-03-04T18:05:33.316+0000 7f7746951640 10 monclient: get_auth_request con 0x55871b0f8580 auth_method 0 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -107> 2024-03-04T18:05:33.316+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -106> 2024-03-04T18:05:33.316+0000 7f774963a980 20 searching for the correct realm 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -105> 2024-03-04T18:05:33.320+0000 7f7747953640 10 monclient: get_auth_request con 0x55871b0fd210 auth_method 0 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -104> 2024-03-04T18:05:33.320+0000 7f7747152640 10 monclient: get_auth_request con 0x7f7740057ca0 auth_method 0 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -103> 2024-03-04T18:05:33.320+0000 7f7746951640 10 monclient: get_auth_request con 0x7f77381109e0 auth_method 0 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -102> 2024-03-04T18:05:33.320+0000 7f7746951640 0 -- 172.21.15.137:0/3100790417 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f77381109e0 msgr2=0x7f7738001a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -101> 2024-03-04T18:05:33.524+0000 7f7746951640 10 monclient: get_auth_request con 0x7f77381109e0 auth_method 0 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -100> 2024-03-04T18:05:33.524+0000 7f7747953640 10 monclient: get_auth_request con 0x7f774005fc10 auth_method 0 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -99> 2024-03-04T18:05:33.532+0000 7f7747152640 10 monclient: get_auth_request con 0x7f77380c2ba0 auth_method 0 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -98> 2024-03-04T18:05:33.532+0000 7f7746951640 0 -- 172.21.15.137:0/3100790417 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x55871b0f8580 msgr2=0x55871b0fa970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -97> 2024-03-04T18:05:33.532+0000 7f76297fa640 1 client.6933.objecter ms_handle_reset 0x55871b0f8580 session 0x55871b0f4330 osd.3 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -96> 2024-03-04T18:05:33.532+0000 7f76297fa640 10 monclient: _renew_subs 2024-03-04T18:05:33.706 INFO:tasks.workunit.client.0.smithi137.stderr: -95> 2024-03-04T18:05:33.532+0000 7f76297fa640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.137:3300/0 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -94> 2024-03-04T18:05:33.532+0000 7f7746951640 10 monclient: get_auth_request con 0x7f76100754c0 auth_method 0 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -93> 2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -92> 2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -91> 2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -90> 2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -89> 2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -88> 2024-03-04T18:05:33.548+0000 7f774963a980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -87> 2024-03-04T18:05:33.548+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -86> 2024-03-04T18:05:33.548+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -85> 2024-03-04T18:05:33.548+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -84> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -83> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -82> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -81> 2024-03-04T18:05:33.552+0000 7f774963a980 20 zone default found 2024-03-04T18:05:33.707 INFO:tasks.workunit.client.0.smithi137.stderr: -80> 2024-03-04T18:05:33.552+0000 7f774963a980 4 Realm: () 2024-03-04T18:05:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: -79> 2024-03-04T18:05:33.552+0000 7f774963a980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:05:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: -78> 2024-03-04T18:05:33.552+0000 7f774963a980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:05:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: -77> 2024-03-04T18:05:33.552+0000 7f774963a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: -76> 2024-03-04T18:05:33.552+0000 7f774963a980 20 zonegroup default 2024-03-04T18:05:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: -75> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: -74> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: -73> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: -72> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: -71> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: -70> 2024-03-04T18:05:33.552+0000 7f774963a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: -69> 2024-03-04T18:05:33.552+0000 7f774963a980 20 started sync module instance, tier type = 2024-03-04T18:05:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: -68> 2024-03-04T18:05:33.552+0000 7f774963a980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:05:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: -67> 2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=0 2024-03-04T18:05:33.708 INFO:tasks.workunit.client.0.smithi137.stderr: -66> 2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=1 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -65> 2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=2 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -64> 2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=3 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -63> 2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=4 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -62> 2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=5 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -61> 2024-03-04T18:05:33.576+0000 7f774963a980 20 add_watcher() i=6 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -60> 2024-03-04T18:05:33.580+0000 7f774963a980 20 add_watcher() i=7 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -59> 2024-03-04T18:05:33.580+0000 7f774963a980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -58> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding auth protocol: cephx 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -57> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding auth protocol: cephx 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -56> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding auth protocol: cephx 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -55> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: secure 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -54> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: crc 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -53> 2024-03-04T18:05:33.584+0000 7f75fe7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -52> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: secure 2024-03-04T18:05:33.709 INFO:tasks.workunit.client.0.smithi137.stderr: -51> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: crc 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -50> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: secure 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -49> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: crc 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -48> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: crc 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -47> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: secure 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -46> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: crc 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -45> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: secure 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -44> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: crc 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -43> 2024-03-04T18:05:33.584+0000 7f774963a980 5 AuthRegistry(0x7ffe8148fb70) adding con mode: secure 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -42> 2024-03-04T18:05:33.584+0000 7f774963a980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -41> 2024-03-04T18:05:33.584+0000 7f774963a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -40> 2024-03-04T18:05:33.584+0000 7f774963a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -39> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command cache list hook 0x55871b0c9460 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -38> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command cache inspect hook 0x55871b0c9460 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -37> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command cache erase hook 0x55871b0c9460 2024-03-04T18:05:33.710 INFO:tasks.workunit.client.0.smithi137.stderr: -36> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command cache zap hook 0x55871b0c9460 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -35> 2024-03-04T18:05:33.584+0000 7f774963a980 5 note: GC not initialized 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -34> 2024-03-04T18:05:33.584+0000 7f75fd7e2640 20 reqs_thread_entry: start 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -33> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command sync trace show hook 0x7f77380043b0 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -32> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command sync trace history hook 0x7f77380043b0 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -31> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command sync trace active hook 0x7f77380043b0 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -30> 2024-03-04T18:05:33.584+0000 7f774963a980 5 asok(0x55871ae20130) register_command sync trace active_short hook 0x7f77380043b0 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -29> 2024-03-04T18:05:33.584+0000 7f774963a980 20 init_complete bucket index max shards: 1 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -28> 2024-03-04T18:05:33.584+0000 7f774963a980 20 Filter name: none 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -27> 2024-03-04T18:05:33.584+0000 7f75f67fc640 20 reqs_thread_entry: start 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -26> 2024-03-04T18:05:33.588+0000 7f774963a980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -25> 2024-03-04T18:05:33.588+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -24> 2024-03-04T18:05:33.588+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -23> 2024-03-04T18:05:33.588+0000 7f774963a980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -22> 2024-03-04T18:05:33.588+0000 7f774963a980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-03-04T18:05:33.711 INFO:tasks.workunit.client.0.smithi137.stderr: -21> 2024-03-04T18:05:33.588+0000 7f774963a980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 : miss 2024-03-04T18:05:33.712 INFO:tasks.workunit.client.0.smithi137.stderr: -20> 2024-03-04T18:05:33.588+0000 7f774963a980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.712 INFO:tasks.workunit.client.0.smithi137.stderr: -19> 2024-03-04T18:05:33.588+0000 7f774963a980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:05:33.712 INFO:tasks.workunit.client.0.smithi137.stderr: -18> 2024-03-04T18:05:33.588+0000 7f774963a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 info.flags=0x17 2024-03-04T18:05:33.712 INFO:tasks.workunit.client.0.smithi137.stderr: -17> 2024-03-04T18:05:33.588+0000 7f774963a980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 to cache LRU end 2024-03-04T18:05:33.712 INFO:tasks.workunit.client.0.smithi137.stderr: -16> 2024-03-04T18:05:33.588+0000 7f774963a980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:05:33.712 INFO:tasks.workunit.client.0.smithi137.stderr: -15> 2024-03-04T18:05:33.588+0000 7f774963a980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:33.712 INFO:tasks.workunit.client.0.smithi137.stderr: -14> 2024-03-04T18:05:33.588+0000 7f774963a980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 2024-03-04T18:05:33.712 INFO:tasks.workunit.client.0.smithi137.stderr: -13> 2024-03-04T18:05:33.596+0000 7f774963a980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 info.flags=0x17 2024-03-04T18:05:33.712 INFO:tasks.workunit.client.0.smithi137.stderr: -12> 2024-03-04T18:05:33.596+0000 7f774963a980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 to cache LRU end 2024-03-04T18:05:33.712 INFO:tasks.workunit.client.0.smithi137.stderr: -11> 2024-03-04T18:05:33.596+0000 7f774963a980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:33.712 INFO:tasks.workunit.client.0.smithi137.stderr: -10> 2024-03-04T18:05:33.596+0000 7f774963a980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29, ofs0, ns] 2024-03-04T18:05:33.712 INFO:tasks.workunit.client.0.smithi137.stderr: -9> 2024-03-04T18:05:33.596+0000 7f762a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627796 cookie 94038762984912 notifier 6933 bl.length()=945 2024-03-04T18:05:33.712 INFO:tasks.workunit.client.0.smithi137.stderr: -8> 2024-03-04T18:05:33.596+0000 7f762a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 info.flags=0x17 2024-03-04T18:05:33.712 INFO:tasks.workunit.client.0.smithi137.stderr: -7> 2024-03-04T18:05:33.596+0000 7f762a7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 to cache LRU end 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr: -6> 2024-03-04T18:05:33.596+0000 7f762a7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr: -5> 2024-03-04T18:05:33.600+0000 7f774963a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): orig_sources= new_sources= 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr: -4> 2024-03-04T18:05:33.600+0000 7f774963a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): potential sources added=[] removed=[] 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr: -3> 2024-03-04T18:05:33.600+0000 7f774963a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): orig_dests= new_dests= 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr: -2> 2024-03-04T18:05:33.600+0000 7f774963a980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): potential dests added=[] removed=[] 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr: -1> 2024-03-04T18:05:33.604+0000 7f774963a980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f774963a980 time 2024-03-04T18:05:33.604827+0000 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f774cf0fd7b] 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f774cf0ff2f] 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558719a10281] 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x558719a1cd5d] 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr: 5: main() 2024-03-04T18:05:33.713 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f774bd7ad90] 2024-03-04T18:05:33.714 INFO:tasks.workunit.client.0.smithi137.stderr: 7: __libc_start_main() 2024-03-04T18:05:33.714 INFO:tasks.workunit.client.0.smithi137.stderr: 8: _start() 2024-03-04T18:05:33.714 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.714 INFO:tasks.workunit.client.0.smithi137.stderr: 0> 2024-03-04T18:05:33.604+0000 7f774963a980 -1 *** Caught signal (Aborted) ** 2024-03-04T18:05:33.714 INFO:tasks.workunit.client.0.smithi137.stderr: in thread 7f774963a980 thread_name:radosgw-admin 2024-03-04T18:05:33.714 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.714 INFO:tasks.workunit.client.0.smithi137.stderr: ceph version 19.0.0-1871-g3aec3903 (3aec390336fdfdc8f927818b718cfb0361bd47c8) squid (dev) 2024-03-04T18:05:33.714 INFO:tasks.workunit.client.0.smithi137.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f774bd93520] 2024-03-04T18:05:33.714 INFO:tasks.workunit.client.0.smithi137.stderr: 2: pthread_kill() 2024-03-04T18:05:33.714 INFO:tasks.workunit.client.0.smithi137.stderr: 3: raise() 2024-03-04T18:05:33.714 INFO:tasks.workunit.client.0.smithi137.stderr: 4: abort() 2024-03-04T18:05:33.714 INFO:tasks.workunit.client.0.smithi137.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f774cf0fdd5] 2024-03-04T18:05:33.714 INFO:tasks.workunit.client.0.smithi137.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x180f2f) [0x7f774cf0ff2f] 2024-03-04T18:05:33.714 INFO:tasks.workunit.client.0.smithi137.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558719a10281] 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x30d) [0x558719a1cd5d] 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr: 9: main() 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f774bd7ad90] 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr: 11: __libc_start_main() 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr: 12: _start() 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr:--- logging levels --- 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 none 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 lockdep 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 context 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 crush 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_balancer 2024-03-04T18:05:33.715 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_locker 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_log_expire 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mds_migrator 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 buffer 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 timer 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 filer 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 striper 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 1 objecter 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rados 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_mirror 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_replay 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 rbd_pwl 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 journaler 2024-03-04T18:05:33.716 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objectcacher 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 immutable_obj_cache 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 client 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 osd 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 optracker 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 objclass 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 filestore 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 journal 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 ms 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mon 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 0/10 monc 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 paxos 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 tp 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 auth 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 crypto 2024-03-04T18:05:33.717 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 finisher 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 reserver 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 heartbeatmap 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 perfcounter 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 20/20 rgw 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_sync 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_datacache 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_access 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_dbstore 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_flight 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 rgw_lifecycle 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 javaclient 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 asok 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 1 throttle 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 0 refs 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 compressor 2024-03-04T18:05:33.718 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluestore 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 bluefs 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 3 bdev 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 kstore 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 4/ 5 rocksdb 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 fuse 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 2/ 5 mgr 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mgrc 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 dpdk 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 eventtrace 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 prioritycache 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 test 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephfs_mirror 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cephsqlite 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 crimson_interrupt 2024-03-04T18:05:33.719 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_onode 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_odata 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_omap 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_tm 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_t 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cleaner 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_epm 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_lba 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_fixedkv_tree 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_cache 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_journal 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_device 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 seastore_backref 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 alienstore 2024-03-04T18:05:33.720 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 mclock 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: 0/ 5 cyanstore 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 ceph_exporter 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 memstore 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: 1/ 5 trace 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: -2/-2 (syslog threshold) 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: 99/99 (stderr threshold) 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr:--- pthread ID / name mapping for recent threads --- 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: 7f75f67fc640 / http_manager 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: 7f75fd7e2640 / http_manager 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: 7f75fe7e4640 / rgw_dt_lg_renew 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: 7f76297fa640 / ms_dispatch 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: 7f762a7fc640 / io_context_pool 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: 7f762affd640 / io_context_pool 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: 7f7631602640 / ms_dispatch 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: 7f7633e07640 / ms_dispatch 2024-03-04T18:05:33.721 INFO:tasks.workunit.client.0.smithi137.stderr: 7f763560a640 / io_context_pool 2024-03-04T18:05:33.722 INFO:tasks.workunit.client.0.smithi137.stderr: 7f7746150640 / io_context_pool 2024-03-04T18:05:33.722 INFO:tasks.workunit.client.0.smithi137.stderr: 7f7746951640 / msgr-worker-2 2024-03-04T18:05:33.722 INFO:tasks.workunit.client.0.smithi137.stderr: 7f7747152640 / msgr-worker-1 2024-03-04T18:05:33.722 INFO:tasks.workunit.client.0.smithi137.stderr: 7f7747953640 / msgr-worker-0 2024-03-04T18:05:33.722 INFO:tasks.workunit.client.0.smithi137.stderr: 7f774963a980 / radosgw-admin 2024-03-04T18:05:33.722 INFO:tasks.workunit.client.0.smithi137.stderr: max_recent 500 2024-03-04T18:05:33.722 INFO:tasks.workunit.client.0.smithi137.stderr: max_new 1000 2024-03-04T18:05:33.722 INFO:tasks.workunit.client.0.smithi137.stderr: log_file /var/lib/ceph/crash/2024-03-04T18:05:33.606827Z_c7ef1ae2-d4e3-4239-b6c0-1c7db4f731e6/log 2024-03-04T18:05:33.722 INFO:tasks.workunit.client.0.smithi137.stderr:--- end dump of recent events --- 2024-03-04T18:05:33.722 INFO:tasks.workunit.client.0.smithi137.stderr:Aborted 2024-03-04T18:05:33.723 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-03-04T18:05:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:05:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:05:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.688+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.688+0000 7f11363dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.688+0000 7f11363dd980 20 realm 2024-03-04T18:05:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.688+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.688+0000 7f11363dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.688+0000 7f11363dd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:05:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.688+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.688+0000 7f11363dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.688+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.692+0000 7f11363dd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.692+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.724 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.692+0000 7f11363dd980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:05:33.725 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.692+0000 7f11363dd980 20 searching for the correct realm 2024-03-04T18:05:33.725 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.696+0000 7f11346f6640 0 -- 172.21.15.137:0/3395121065 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f112c029950 msgr2=0x7f112c038410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:33.910 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.912+0000 7f11346f6640 0 -- 172.21.15.137:0/3395121065 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x560611693950 msgr2=0x560611695e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:05:33.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.920+0000 7f11363dd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:05:33.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.920+0000 7f11363dd980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:05:33.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.920+0000 7f11363dd980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:05:33.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.920+0000 7f11363dd980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:05:33.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.920+0000 7f11363dd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:05:33.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.920+0000 7f11363dd980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:05:33.919 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.920+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.920+0000 7f11363dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.920+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:05:33.920 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:05:33.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 20 zone default found 2024-03-04T18:05:33.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 4 Realm: () 2024-03-04T18:05:33.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:05:33.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:05:33.921 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:05:33.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 20 zonegroup default 2024-03-04T18:05:33.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.922 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 20 started sync module instance, tier type = 2024-03-04T18:05:33.923 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.924+0000 7f11363dd980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:05:33.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.948+0000 7f11363dd980 20 add_watcher() i=0 2024-03-04T18:05:33.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.948+0000 7f11363dd980 20 add_watcher() i=1 2024-03-04T18:05:33.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.948+0000 7f11363dd980 20 add_watcher() i=2 2024-03-04T18:05:33.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.948+0000 7f11363dd980 20 add_watcher() i=3 2024-03-04T18:05:33.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.948+0000 7f11363dd980 20 add_watcher() i=4 2024-03-04T18:05:33.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.948+0000 7f11363dd980 20 add_watcher() i=5 2024-03-04T18:05:33.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.948+0000 7f11363dd980 20 add_watcher() i=6 2024-03-04T18:05:33.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.948+0000 7f11363dd980 20 add_watcher() i=7 2024-03-04T18:05:33.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.948+0000 7f11363dd980 2 all 8 watchers are set, enabling cache 2024-03-04T18:05:33.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.952+0000 7f0fe9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:33.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.952+0000 7f11363dd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:05:33.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.956+0000 7f11363dd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:05:33.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.956+0000 7f11363dd980 5 note: GC not initialized 2024-03-04T18:05:33.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.956+0000 7f0fe8fe1640 20 reqs_thread_entry: start 2024-03-04T18:05:33.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.956+0000 7f11363dd980 20 init_complete bucket index max shards: 1 2024-03-04T18:05:33.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.956+0000 7f11363dd980 20 Filter name: none 2024-03-04T18:05:33.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.956+0000 7f0f627fc640 20 reqs_thread_entry: start 2024-03-04T18:05:33.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.956+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.953 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.956+0000 7f11363dd980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:33.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.956+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.956+0000 7f11363dd980 20 rados_obj.operate() r=0 bl.length=415 2024-03-04T18:05:33.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.956+0000 7f11363dd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:33.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.956+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.956+0000 7f11363dd980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:05:33.955 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.956+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.960+0000 7f11363dd980 20 rados_obj.operate() r=0 bl.length=415 2024-03-04T18:05:33.957 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.960+0000 7f11363dd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:05:33.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.960+0000 7f11363dd980 20 rados->read ofs=0 len=0 2024-03-04T18:05:33.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.960+0000 7f11363dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:05:33.959 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.960+0000 7f0fe9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:05:33.963 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.964+0000 7f11363dd980 20 remove_watcher() i=0 2024-03-04T18:05:33.963 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.964+0000 7f11363dd980 2 removed watcher, disabling cache 2024-03-04T18:05:33.965 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.968+0000 7f11363dd980 20 remove_watcher() i=1 2024-03-04T18:05:33.967 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.968+0000 7f11363dd980 20 remove_watcher() i=2 2024-03-04T18:05:33.969 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.972+0000 7f11363dd980 20 remove_watcher() i=3 2024-03-04T18:05:33.971 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.972+0000 7f11363dd980 20 remove_watcher() i=4 2024-03-04T18:05:33.973 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.976+0000 7f11363dd980 20 remove_watcher() i=5 2024-03-04T18:05:33.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.980+0000 7f11363dd980 20 remove_watcher() i=6 2024-03-04T18:05:33.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:05:33.980+0000 7f11363dd980 20 remove_watcher() i=7 2024-03-04T18:05:33.994 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.994 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:05:33.994 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: abort-at-do-reshard id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 num_objects: 20 size_kb: 1 num_shards: 1 2024-03-04T18:05:33.994 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:05:33.995 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:05:33.995 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:04.169 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:04.169 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:04.250 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-03-04T18:06:04.285 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:04.285 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:04.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.331+0000 7faa2d7dc980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.331+0000 7faa2d7dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.331+0000 7faa2d7dc980 20 realm 2024-03-04T18:06:04.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.331+0000 7faa2d7dc980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.331+0000 7faa2d7dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.331+0000 7faa2d7dc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:04.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.331+0000 7faa2d7dc980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.331+0000 7faa2d7dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.331+0000 7faa2d7dc980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.335+0000 7faa2d7dc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:04.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.335+0000 7faa2d7dc980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.335+0000 7faa2d7dc980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:04.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.335+0000 7faa2d7dc980 20 searching for the correct realm 2024-03-04T18:06:04.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.339+0000 7faa2baf5640 0 -- 172.21.15.137:0/2873836550 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7faa2400e040 msgr2=0x7faa2401e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:04.554 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.555+0000 7faa2baf5640 0 -- 172.21.15.137:0/2873836550 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x56304fbf92d0 msgr2=0x56304fbfb790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:04.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.563+0000 7faa2d7dc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:04.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.563+0000 7faa2d7dc980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:04.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.563+0000 7faa2d7dc980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:04.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.563+0000 7faa2d7dc980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:04.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.563+0000 7faa2d7dc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:04.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.563+0000 7faa2d7dc980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:04.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.563+0000 7faa2d7dc980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.563+0000 7faa2d7dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.563+0000 7faa2d7dc980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:04.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:04.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 20 zone default found 2024-03-04T18:06:04.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 4 Realm: () 2024-03-04T18:06:04.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:04.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:04.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:04.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 20 zonegroup default 2024-03-04T18:06:04.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 20 started sync module instance, tier type = 2024-03-04T18:06:04.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.567+0000 7faa2d7dc980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:04.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.591+0000 7faa2d7dc980 20 add_watcher() i=0 2024-03-04T18:06:04.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.591+0000 7faa2d7dc980 20 add_watcher() i=1 2024-03-04T18:06:04.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.591+0000 7faa2d7dc980 20 add_watcher() i=2 2024-03-04T18:06:04.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.591+0000 7faa2d7dc980 20 add_watcher() i=3 2024-03-04T18:06:04.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.591+0000 7faa2d7dc980 20 add_watcher() i=4 2024-03-04T18:06:04.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.591+0000 7faa2d7dc980 20 add_watcher() i=5 2024-03-04T18:06:04.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.591+0000 7faa2d7dc980 20 add_watcher() i=6 2024-03-04T18:06:04.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.595+0000 7faa2d7dc980 20 add_watcher() i=7 2024-03-04T18:06:04.593 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.595+0000 7faa2d7dc980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:04.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.599+0000 7fa8e1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:04.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.599+0000 7faa2d7dc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:04.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.599+0000 7faa2d7dc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:04.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.599+0000 7faa2d7dc980 5 note: GC not initialized 2024-03-04T18:06:04.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.599+0000 7fa8e0fe1640 20 reqs_thread_entry: start 2024-03-04T18:06:04.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.599+0000 7faa2d7dc980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:04.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.599+0000 7faa2d7dc980 20 Filter name: none 2024-03-04T18:06:04.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.599+0000 7fa85a7fc640 20 reqs_thread_entry: start 2024-03-04T18:06:04.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.599+0000 7faa2d7dc980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-03-04T18:06:04.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.599+0000 7faa2d7dc980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.601 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.603+0000 7faa2d7dc980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:06:04.601 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.603+0000 7faa2d7dc980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-03-04T18:06:04.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.603+0000 7faa2d7dc980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-03-04T18:06:04.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.603+0000 7faa2d7dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 : miss 2024-03-04T18:06:04.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.603+0000 7faa2d7dc980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.603+0000 7faa2d7dc980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:06:04.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.603+0000 7faa2d7dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 info.flags=0x17 2024-03-04T18:06:04.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.603+0000 7faa2d7dc980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 to cache LRU end 2024-03-04T18:06:04.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.603+0000 7faa2d7dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:04.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.603+0000 7faa2d7dc980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:06:04.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.603+0000 7faa2d7dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 2024-03-04T18:06:04.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.611+0000 7faa2d7dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 info.flags=0x17 2024-03-04T18:06:04.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.611+0000 7faa2d7dc980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 to cache LRU end 2024-03-04T18:06:04.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.611+0000 7faa2d7dc980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:06:04.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.611+0000 7faa2d7dc980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29, ofs0, ns] 2024-03-04T18:06:04.613 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.615+0000 7fa90e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627798 cookie 94765496394016 notifier 6587 bl.length()=945 2024-03-04T18:06:04.613 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.615+0000 7fa90e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 info.flags=0x17 2024-03-04T18:06:04.613 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.615+0000 7fa90e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 to cache LRU end 2024-03-04T18:06:04.613 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.615+0000 7fa90e7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:06:04.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.615+0000 7faa2d7dc980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): orig_sources= new_sources= 2024-03-04T18:06:04.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.615+0000 7faa2d7dc980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): potential sources added=[] removed=[] 2024-03-04T18:06:04.615 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.615+0000 7faa2d7dc980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): orig_dests= new_dests= 2024-03-04T18:06:04.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.615+0000 7faa2d7dc980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): potential dests added=[] removed=[] 2024-03-04T18:06:04.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.619+0000 7faa2d7dc980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29.1.0 2024-03-04T18:06:04.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.619+0000 7faa2d7dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29.1.0 2024-03-04T18:06:04.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.619+0000 7faa2d7dc980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29.1.1 2024-03-04T18:06:04.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.619+0000 7faa2d7dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29.1.1 2024-03-04T18:06:04.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.619+0000 7faa2d7dc980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29.0 2024-03-04T18:06:04.625 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.627+0000 7faa2d7dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 : hit (requested=0x17, cached=0x17) 2024-03-04T18:06:04.625 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.627+0000 7faa2d7dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 2024-03-04T18:06:04.628 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.627+0000 7faa2d7dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 info.flags=0x17 2024-03-04T18:06:04.628 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.627+0000 7faa2d7dc980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 to cache LRU end 2024-03-04T18:06:04.628 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.627+0000 7faa2d7dc980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:06:04.628 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.627+0000 7faa2d7dc980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29, ofs0, ns] 2024-03-04T18:06:04.628 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.631+0000 7fa90effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627799 cookie 94765496394016 notifier 6587 bl.length()=913 2024-03-04T18:06:04.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.631+0000 7fa90effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 info.flags=0x17 2024-03-04T18:06:04.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.631+0000 7fa90effd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 to cache LRU end 2024-03-04T18:06:04.629 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.631+0000 7fa90effd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-03-04T18:06:04.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.631+0000 7faa2d7dc980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): orig_sources= new_sources= 2024-03-04T18:06:04.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.631+0000 7faa2d7dc980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): potential sources added=[] removed=[] 2024-03-04T18:06:04.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.631+0000 7faa2d7dc980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): orig_dests= new_dests= 2024-03-04T18:06:04.631 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.631+0000 7faa2d7dc980 20 handle_bi_update(): bucket=:abort-at-do-reshard[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29]): potential dests added=[] removed=[] 2024-03-04T18:06:04.635 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.635+0000 7faa2d7dc980 1 execute INFO: reshard of bucket "abort-at-do-reshard" completed successfully 2024-03-04T18:06:04.638 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.639+0000 7fa8e1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:04.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.643+0000 7faa2d7dc980 20 remove_watcher() i=0 2024-03-04T18:06:04.642 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.643+0000 7faa2d7dc980 2 removed watcher, disabling cache 2024-03-04T18:06:04.644 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.647+0000 7faa2d7dc980 20 remove_watcher() i=1 2024-03-04T18:06:04.646 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.647+0000 7faa2d7dc980 20 remove_watcher() i=2 2024-03-04T18:06:04.648 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.651+0000 7faa2d7dc980 20 remove_watcher() i=3 2024-03-04T18:06:04.650 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.651+0000 7faa2d7dc980 20 remove_watcher() i=4 2024-03-04T18:06:04.652 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.655+0000 7faa2d7dc980 20 remove_watcher() i=5 2024-03-04T18:06:04.655 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.655+0000 7faa2d7dc980 20 remove_watcher() i=6 2024-03-04T18:06:04.658 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.659+0000 7faa2d7dc980 20 remove_watcher() i=7 2024-03-04T18:06:04.674 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-03-04T18:06:04.705 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:04.705 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:04.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.751+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.751+0000 7f2fd7134980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.751+0000 7f2fd7134980 20 realm 2024-03-04T18:06:04.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.751+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.751+0000 7f2fd7134980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.751+0000 7f2fd7134980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:04.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.751+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.751+0000 7f2fd7134980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.751+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.755+0000 7f2fd7134980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:04.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.755+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.755+0000 7f2fd7134980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:04.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.755+0000 7f2fd7134980 20 searching for the correct realm 2024-03-04T18:06:04.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.759+0000 7f2fd544d640 0 -- 172.21.15.137:0/3303270836 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f2fd00590b0 msgr2=0x7f2fd0069790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:04.975 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.975+0000 7f2fd544d640 0 -- 172.21.15.137:0/3303270836 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x557c666514a0 msgr2=0x557c66653960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:04.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:04.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:04.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:04.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:04.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:04.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:04.984 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.985 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.985 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.985 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:04.985 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:04.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 zone default found 2024-03-04T18:06:04.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 4 Realm: () 2024-03-04T18:06:04.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:04.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:04.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:04.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 zonegroup default 2024-03-04T18:06:04.986 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.987 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.987 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.988 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.988 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.987+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:04.988 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.991+0000 7f2fd7134980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:04.988 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.991+0000 7f2fd7134980 20 started sync module instance, tier type = 2024-03-04T18:06:04.989 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:04.991+0000 7f2fd7134980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:05.012 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.011+0000 7f2fd7134980 20 add_watcher() i=0 2024-03-04T18:06:05.012 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.011+0000 7f2fd7134980 20 add_watcher() i=1 2024-03-04T18:06:05.012 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.011+0000 7f2fd7134980 20 add_watcher() i=2 2024-03-04T18:06:05.012 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.011+0000 7f2fd7134980 20 add_watcher() i=3 2024-03-04T18:06:05.012 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.011+0000 7f2fd7134980 20 add_watcher() i=4 2024-03-04T18:06:05.012 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.011+0000 7f2fd7134980 20 add_watcher() i=5 2024-03-04T18:06:05.012 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.011+0000 7f2fd7134980 20 add_watcher() i=6 2024-03-04T18:06:05.014 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.015+0000 7f2fd7134980 20 add_watcher() i=7 2024-03-04T18:06:05.015 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.015+0000 7f2fd7134980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:05.019 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.019+0000 7f2e8afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:05.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.023+0000 7f2fd7134980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:05.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.023+0000 7f2fd7134980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:05.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.023+0000 7f2fd7134980 5 note: GC not initialized 2024-03-04T18:06:05.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.023+0000 7f2e89fe3640 20 reqs_thread_entry: start 2024-03-04T18:06:05.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.023+0000 7f2fd7134980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:05.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.023+0000 7f2fd7134980 20 Filter name: none 2024-03-04T18:06:05.021 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.023+0000 7f2dfb7fe640 20 reqs_thread_entry: start 2024-03-04T18:06:05.022 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.023+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.022 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.023+0000 7f2fd7134980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:06:05.022 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.023+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.024 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.027+0000 7f2fd7134980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:06:05.024 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.027+0000 7f2fd7134980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:05.024 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.027+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.027+0000 7f2fd7134980 20 rados_obj.operate() r=0 bl.length=211 2024-03-04T18:06:05.025 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.027+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.026 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.027+0000 7f2fd7134980 20 rados_obj.operate() r=0 bl.length=383 2024-03-04T18:06:05.026 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.027+0000 7f2fd7134980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:05.027 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.027+0000 7f2fd7134980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.028 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.031+0000 7f2fd7134980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.029 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.031+0000 7f2e8afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:05.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.035+0000 7f2fd7134980 20 remove_watcher() i=0 2024-03-04T18:06:05.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.035+0000 7f2fd7134980 2 removed watcher, disabling cache 2024-03-04T18:06:05.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.039+0000 7f2fd7134980 20 remove_watcher() i=1 2024-03-04T18:06:05.038 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.039+0000 7f2fd7134980 20 remove_watcher() i=2 2024-03-04T18:06:05.040 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.043+0000 7f2fd7134980 20 remove_watcher() i=3 2024-03-04T18:06:05.042 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.043+0000 7f2fd7134980 20 remove_watcher() i=4 2024-03-04T18:06:05.044 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.047+0000 7f2fd7134980 20 remove_watcher() i=5 2024-03-04T18:06:05.047 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.047+0000 7f2fd7134980 20 remove_watcher() i=6 2024-03-04T18:06:05.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.051+0000 7f2fd7134980 20 remove_watcher() i=7 2024-03-04T18:06:05.065 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:06:05.065 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:06:05.065 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: abort-at-do-reshard id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.29 num_objects: 19 size_kb: 1 num_shards: 2 2024-03-04T18:06:05.065 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:06:05.067 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:05.067 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:05.085 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:05.085 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:05.427 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:05.428 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:05.593 INFO:tasks.workunit.client.0.smithi137.stderr: test: reshard versioned bucket 2024-03-04T18:06:05.594 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket myver 2024-03-04T18:06:05.628 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:05.628 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:05.668 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.671+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.671+0000 7f84a3245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.671+0000 7f84a3245980 20 realm 2024-03-04T18:06:05.670 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.671+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.671+0000 7f84a3245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.671+0000 7f84a3245980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:05.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.671+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.671+0000 7f84a3245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.671 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.671+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.672 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.675+0000 7f84a3245980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:05.672 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.675+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.675+0000 7f84a3245980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:05.673 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.675+0000 7f84a3245980 20 searching for the correct realm 2024-03-04T18:06:05.678 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.679+0000 7f84a155e640 0 -- 172.21.15.137:0/2541032477 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f84980c0070 msgr2=0x7f84980cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:05.893 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.895+0000 7f84a155e640 0 -- 172.21.15.137:0/2541032477 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55786ebab740 msgr2=0x55786ebadbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:05.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.903+0000 7f84a3245980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:05.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.903+0000 7f84a3245980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:05.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.903+0000 7f84a3245980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:05.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.903+0000 7f84a3245980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:05.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.903+0000 7f84a3245980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:05.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.903+0000 7f84a3245980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:05.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.903+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.904 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.903+0000 7f84a3245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.904 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.903+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.904 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:05.905 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:05.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 20 zone default found 2024-03-04T18:06:05.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 4 Realm: () 2024-03-04T18:06:05.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:05.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:05.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:05.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 20 zonegroup default 2024-03-04T18:06:05.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.908 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 20 started sync module instance, tier type = 2024-03-04T18:06:05.908 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.907+0000 7f84a3245980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:05.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.931+0000 7f84a3245980 20 add_watcher() i=0 2024-03-04T18:06:05.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.931+0000 7f84a3245980 20 add_watcher() i=1 2024-03-04T18:06:05.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.931+0000 7f84a3245980 20 add_watcher() i=2 2024-03-04T18:06:05.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.931+0000 7f84a3245980 20 add_watcher() i=3 2024-03-04T18:06:05.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.931+0000 7f84a3245980 20 add_watcher() i=4 2024-03-04T18:06:05.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.931+0000 7f84a3245980 20 add_watcher() i=5 2024-03-04T18:06:05.931 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.931+0000 7f84a3245980 20 add_watcher() i=6 2024-03-04T18:06:05.933 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.935+0000 7f84a3245980 20 add_watcher() i=7 2024-03-04T18:06:05.933 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.935+0000 7f84a3245980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:05.937 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.939+0000 7f8356fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:05.938 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.939+0000 7f84a3245980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:05.938 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.939+0000 7f84a3245980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:05.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.939+0000 7f84a3245980 5 note: GC not initialized 2024-03-04T18:06:05.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.939+0000 7f8355fdb640 20 reqs_thread_entry: start 2024-03-04T18:06:05.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.939+0000 7f84a3245980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:05.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.939+0000 7f84a3245980 20 Filter name: none 2024-03-04T18:06:05.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.939+0000 7f82c77fe640 20 reqs_thread_entry: start 2024-03-04T18:06:05.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.943+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.943+0000 7f84a3245980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:05.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.943+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.943+0000 7f84a3245980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:05.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.943+0000 7f84a3245980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:05.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.943+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.943+0000 7f84a3245980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:05.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.943+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.947+0000 7f84a3245980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:05.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.947+0000 7f84a3245980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:05.945 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.947+0000 7f84a3245980 20 rados->read ofs=0 len=0 2024-03-04T18:06:05.946 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.947+0000 7f84a3245980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:05.948 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.947+0000 7f8356fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:05.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.955+0000 7f84a3245980 20 remove_watcher() i=0 2024-03-04T18:06:05.952 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.955+0000 7f84a3245980 2 removed watcher, disabling cache 2024-03-04T18:06:05.954 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.955+0000 7f84a3245980 20 remove_watcher() i=1 2024-03-04T18:06:05.956 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.959+0000 7f84a3245980 20 remove_watcher() i=2 2024-03-04T18:06:05.958 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.959+0000 7f84a3245980 20 remove_watcher() i=3 2024-03-04T18:06:05.960 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.963+0000 7f84a3245980 20 remove_watcher() i=4 2024-03-04T18:06:05.962 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.963+0000 7f84a3245980 20 remove_watcher() i=5 2024-03-04T18:06:05.965 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.967+0000 7f84a3245980 20 remove_watcher() i=6 2024-03-04T18:06:05.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:05.971+0000 7f84a3245980 20 remove_watcher() i=7 2024-03-04T18:06:05.982 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:06:05.982 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:06:05.982 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: myver id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 num_objects: 0 size_kb: 0 num_shards: 1 2024-03-04T18:06:05.983 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:06:05.983 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 2 2024-03-04T18:06:06.016 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:06.017 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:06.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.059+0000 7f47fc738980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.063+0000 7f47fc738980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.063+0000 7f47fc738980 20 realm 2024-03-04T18:06:06.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.063+0000 7f47fc738980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.063+0000 7f47fc738980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.063+0000 7f47fc738980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:06.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.063+0000 7f47fc738980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.063+0000 7f47fc738980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.062 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.063+0000 7f47fc738980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.063+0000 7f47fc738980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:06.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.063+0000 7f47fc738980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.063 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.063+0000 7f47fc738980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:06.064 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.063+0000 7f47fc738980 20 searching for the correct realm 2024-03-04T18:06:06.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.071+0000 7f47faa51640 0 -- 172.21.15.137:0/3040840737 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f47ec0b88f0 msgr2=0x7f47ec0aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:06.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.287+0000 7f47faa51640 0 -- 172.21.15.137:0/3040840737 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x56038b331c60 msgr2=0x56038b334120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:06.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.295+0000 7f47fc738980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:06.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.295+0000 7f47fc738980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:06.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.295+0000 7f47fc738980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:06.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.295+0000 7f47fc738980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:06.295 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.295+0000 7f47fc738980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:06.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.295+0000 7f47fc738980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:06.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.295+0000 7f47fc738980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.295+0000 7f47fc738980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.296 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.295+0000 7f47fc738980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:06.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:06.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 20 zone default found 2024-03-04T18:06:06.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 4 Realm: () 2024-03-04T18:06:06.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:06.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:06.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:06.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 20 zonegroup default 2024-03-04T18:06:06.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.298 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.299 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.300 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.300 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 20 started sync module instance, tier type = 2024-03-04T18:06:06.300 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.299+0000 7f47fc738980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:06.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.323+0000 7f47fc738980 20 add_watcher() i=0 2024-03-04T18:06:06.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.323+0000 7f47fc738980 20 add_watcher() i=1 2024-03-04T18:06:06.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.323+0000 7f47fc738980 20 add_watcher() i=2 2024-03-04T18:06:06.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.323+0000 7f47fc738980 20 add_watcher() i=3 2024-03-04T18:06:06.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.323+0000 7f47fc738980 20 add_watcher() i=4 2024-03-04T18:06:06.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.323+0000 7f47fc738980 20 add_watcher() i=5 2024-03-04T18:06:06.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.323+0000 7f47fc738980 20 add_watcher() i=6 2024-03-04T18:06:06.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.327+0000 7f47fc738980 20 add_watcher() i=7 2024-03-04T18:06:06.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.327+0000 7f47fc738980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:06.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.331+0000 7f46b0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:06.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.331+0000 7f47fc738980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:06.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.331+0000 7f47fc738980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:06.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.331+0000 7f47fc738980 5 note: GC not initialized 2024-03-04T18:06:06.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.331+0000 7f462b7fe640 20 reqs_thread_entry: start 2024-03-04T18:06:06.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.331+0000 7f47fc738980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:06.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.331+0000 7f47fc738980 20 Filter name: none 2024-03-04T18:06:06.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.331+0000 7f46297fa640 20 reqs_thread_entry: start 2024-03-04T18:06:06.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.331+0000 7f47fc738980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-03-04T18:06:06.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.331+0000 7f47fc738980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.335+0000 7f47fc738980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:06.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.335+0000 7f47fc738980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-03-04T18:06:06.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.335+0000 7f47fc738980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-03-04T18:06:06.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.335+0000 7f47fc738980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 : miss 2024-03-04T18:06:06.332 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.335+0000 7f47fc738980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.335+0000 7f47fc738980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:06.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.335+0000 7f47fc738980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:06.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.335+0000 7f47fc738980 10 adding default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:06.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.335+0000 7f47fc738980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:06.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.335+0000 7f47fc738980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:06.333 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.335+0000 7f47fc738980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 2024-03-04T18:06:06.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.343+0000 7f47fc738980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:06.341 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.343+0000 7f47fc738980 10 moving default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:06.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.343+0000 7f47fc738980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:06.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.343+0000 7f47fc738980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19, ofs0, ns] 2024-03-04T18:06:06.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.343+0000 7f46ddffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627853 cookie 94573220278976 notifier 7023 bl.length()=855 2024-03-04T18:06:06.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.343+0000 7f46ddffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:06.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.343+0000 7f46ddffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:06.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.343+0000 7f46ddffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:06.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.347+0000 7f47fc738980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): orig_sources= new_sources= 2024-03-04T18:06:06.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.347+0000 7f47fc738980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): potential sources added=[] removed=[] 2024-03-04T18:06:06.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.347+0000 7f47fc738980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): orig_dests= new_dests= 2024-03-04T18:06:06.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.347+0000 7f47fc738980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): potential dests added=[] removed=[] 2024-03-04T18:06:06.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.347+0000 7f47fc738980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.1.0 2024-03-04T18:06:06.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.347+0000 7f47fc738980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.1.0 2024-03-04T18:06:06.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.347+0000 7f47fc738980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.1.1 2024-03-04T18:06:06.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.347+0000 7f47fc738980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.1.1 2024-03-04T18:06:06.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.347+0000 7f47fc738980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.0 2024-03-04T18:06:06.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.347+0000 7f47fc738980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 : hit (requested=0x17, cached=0x17) 2024-03-04T18:06:06.348 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.347+0000 7f47fc738980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 2024-03-04T18:06:06.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.351+0000 7f47fc738980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:06.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.351+0000 7f47fc738980 10 moving default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:06.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.351+0000 7f47fc738980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:06.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.351+0000 7f47fc738980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19, ofs0, ns] 2024-03-04T18:06:06.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.355+0000 7f46ddffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627854 cookie 94573220278976 notifier 7023 bl.length()=823 2024-03-04T18:06:06.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.355+0000 7f46ddffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:06.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.355+0000 7f46ddffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:06.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.355+0000 7f46ddffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:06.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.355+0000 7f47fc738980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): orig_sources= new_sources= 2024-03-04T18:06:06.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.355+0000 7f47fc738980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): potential sources added=[] removed=[] 2024-03-04T18:06:06.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.355+0000 7f47fc738980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): orig_dests= new_dests= 2024-03-04T18:06:06.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.355+0000 7f47fc738980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): potential dests added=[] removed=[] 2024-03-04T18:06:06.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.355+0000 7f47fc738980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-03-04T18:06:06.359 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.359+0000 7f46b0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:06.364 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.363+0000 7f47fc738980 20 remove_watcher() i=0 2024-03-04T18:06:06.364 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.363+0000 7f47fc738980 2 removed watcher, disabling cache 2024-03-04T18:06:06.366 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.367+0000 7f47fc738980 20 remove_watcher() i=1 2024-03-04T18:06:06.368 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.371+0000 7f47fc738980 20 remove_watcher() i=2 2024-03-04T18:06:06.370 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.371+0000 7f47fc738980 20 remove_watcher() i=3 2024-03-04T18:06:06.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.375+0000 7f47fc738980 20 remove_watcher() i=4 2024-03-04T18:06:06.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.375+0000 7f47fc738980 20 remove_watcher() i=5 2024-03-04T18:06:06.377 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.379+0000 7f47fc738980 20 remove_watcher() i=6 2024-03-04T18:06:06.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.383+0000 7f47fc738980 20 remove_watcher() i=7 2024-03-04T18:06:06.397 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket myver 2024-03-04T18:06:06.427 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:06.427 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:06.470 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.471+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.472 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.475+0000 7f26f7375980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.472 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.475+0000 7f26f7375980 20 realm 2024-03-04T18:06:06.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.475+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.475+0000 7f26f7375980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.475+0000 7f26f7375980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:06.473 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.475+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.474 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.475+0000 7f26f7375980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.474 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.475+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.474 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.475+0000 7f26f7375980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:06.474 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.475+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.475+0000 7f26f7375980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:06.475 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.475+0000 7f26f7375980 20 searching for the correct realm 2024-03-04T18:06:06.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.483+0000 7f26f568e640 0 -- 172.21.15.137:0/3239878198 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f26ec01d3a0 msgr2=0x7f26ec01d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:06.695 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.695+0000 7f26f568e640 0 -- 172.21.15.137:0/3239878198 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x55d2bb61a8f0 msgr2=0x55d2bb61cd60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:06.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.707+0000 7f26f7375980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:06.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.707+0000 7f26f7375980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:06.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.707+0000 7f26f7375980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:06.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.707+0000 7f26f7375980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:06.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.707+0000 7f26f7375980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:06.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.707+0000 7f26f7375980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:06.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.707+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:06.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:06.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 20 zone default found 2024-03-04T18:06:06.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 4 Realm: () 2024-03-04T18:06:06.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:06.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:06.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:06.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 20 zonegroup default 2024-03-04T18:06:06.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.711 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.711+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.712 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.715+0000 7f26f7375980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.712 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.715+0000 7f26f7375980 20 started sync module instance, tier type = 2024-03-04T18:06:06.712 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.715+0000 7f26f7375980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:06.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.735+0000 7f26f7375980 20 add_watcher() i=0 2024-03-04T18:06:06.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.735+0000 7f26f7375980 20 add_watcher() i=1 2024-03-04T18:06:06.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.735+0000 7f26f7375980 20 add_watcher() i=2 2024-03-04T18:06:06.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.735+0000 7f26f7375980 20 add_watcher() i=3 2024-03-04T18:06:06.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.735+0000 7f26f7375980 20 add_watcher() i=4 2024-03-04T18:06:06.734 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.735+0000 7f26f7375980 20 add_watcher() i=5 2024-03-04T18:06:06.735 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.735+0000 7f26f7375980 20 add_watcher() i=6 2024-03-04T18:06:06.737 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.739+0000 7f26f7375980 20 add_watcher() i=7 2024-03-04T18:06:06.737 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.739+0000 7f26f7375980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:06.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.743+0000 7f25aafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:06.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.743+0000 7f26f7375980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:06.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.743+0000 7f26f7375980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:06.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.743+0000 7f26f7375980 5 note: GC not initialized 2024-03-04T18:06:06.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.743+0000 7f25a9fe3640 20 reqs_thread_entry: start 2024-03-04T18:06:06.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.743+0000 7f26f7375980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:06.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.743+0000 7f26f7375980 20 Filter name: none 2024-03-04T18:06:06.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.743+0000 7f251b7fe640 20 reqs_thread_entry: start 2024-03-04T18:06:06.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.743+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.747+0000 7f26f7375980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:06.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.747+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.747+0000 7f26f7375980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:06.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.747+0000 7f26f7375980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:06.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.747+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.747 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.747+0000 7f26f7375980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:06.748 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.747+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.749 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.751+0000 7f26f7375980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:06.749 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.751+0000 7f26f7375980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:06.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.751+0000 7f26f7375980 20 rados->read ofs=0 len=0 2024-03-04T18:06:06.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.751+0000 7f26f7375980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:06.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.755+0000 7f25aafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:06.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.759+0000 7f26f7375980 20 remove_watcher() i=0 2024-03-04T18:06:06.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.759+0000 7f26f7375980 2 removed watcher, disabling cache 2024-03-04T18:06:06.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.759+0000 7f26f7375980 20 remove_watcher() i=1 2024-03-04T18:06:06.760 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.763+0000 7f26f7375980 20 remove_watcher() i=2 2024-03-04T18:06:06.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.763+0000 7f26f7375980 20 remove_watcher() i=3 2024-03-04T18:06:06.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.767+0000 7f26f7375980 20 remove_watcher() i=4 2024-03-04T18:06:06.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.767+0000 7f26f7375980 20 remove_watcher() i=5 2024-03-04T18:06:06.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.771+0000 7f26f7375980 20 remove_watcher() i=6 2024-03-04T18:06:06.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:06.775+0000 7f26f7375980 20 remove_watcher() i=7 2024-03-04T18:06:06.796 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:06:06.796 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:06:06.796 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: myver id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 num_objects: 0 size_kb: 0 num_shards: 2 2024-03-04T18:06:06.796 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:06:06.800 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:06.800 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:06.824 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:06.824 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:06.905 INFO:tasks.workunit.client.0.smithi137.stderr: test: reshard removes olh entries with empty name 2024-03-04T18:06:06.910 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:06.910 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:07.025 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:07.026 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:07.242 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-03-04T18:06:07.273 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:07.273 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:07.317 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.319+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.319+0000 7f07a3d5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.319+0000 7f07a3d5b980 20 realm 2024-03-04T18:06:07.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.319+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.320 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.323+0000 7f07a3d5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.320 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.323+0000 7f07a3d5b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:07.320 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.323+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.320 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.323+0000 7f07a3d5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.320 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.323+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.323+0000 7f07a3d5b980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:07.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.323+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.323+0000 7f07a3d5b980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:07.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.323+0000 7f07a3d5b980 20 searching for the correct realm 2024-03-04T18:06:07.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.331+0000 7f07a2074640 0 -- 172.21.15.137:0/126632279 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f079c02b8a0 msgr2=0x7f079c01e3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:07.336 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.339+0000 7f07a2074640 0 -- 172.21.15.137:0/126632279 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x560f87a7fd60 msgr2=0x560f87a82220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:07.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:07.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:07.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:07.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:07.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:07.342 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:07.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:07.343 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:07.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 zone default found 2024-03-04T18:06:07.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 4 Realm: () 2024-03-04T18:06:07.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:07.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:07.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:07.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 zonegroup default 2024-03-04T18:06:07.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.343+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.347+0000 7f07a3d5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.347+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.347+0000 7f07a3d5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.347+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.347+0000 7f07a3d5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.347+0000 7f07a3d5b980 20 started sync module instance, tier type = 2024-03-04T18:06:07.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.347+0000 7f07a3d5b980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:07.363 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.363+0000 7f07a3d5b980 20 add_watcher() i=0 2024-03-04T18:06:07.364 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.363+0000 7f07a3d5b980 20 add_watcher() i=1 2024-03-04T18:06:07.364 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.363+0000 7f07a3d5b980 20 add_watcher() i=2 2024-03-04T18:06:07.364 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.363+0000 7f07a3d5b980 20 add_watcher() i=3 2024-03-04T18:06:07.364 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.363+0000 7f07a3d5b980 20 add_watcher() i=4 2024-03-04T18:06:07.364 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.363+0000 7f07a3d5b980 20 add_watcher() i=5 2024-03-04T18:06:07.364 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.363+0000 7f07a3d5b980 20 add_watcher() i=6 2024-03-04T18:06:07.366 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.367+0000 7f07a3d5b980 20 add_watcher() i=7 2024-03-04T18:06:07.366 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.367+0000 7f07a3d5b980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:07.369 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f06537fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:07.369 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f07a3d5b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:07.369 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f07a3d5b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:07.370 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f07a3d5b980 5 note: GC not initialized 2024-03-04T18:06:07.370 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f06527fc640 20 reqs_thread_entry: start 2024-03-04T18:06:07.370 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f07a3d5b980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:07.370 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f07a3d5b980 20 Filter name: none 2024-03-04T18:06:07.370 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f05c3fff640 20 reqs_thread_entry: start 2024-03-04T18:06:07.370 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.371 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f07a3d5b980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:06:07.371 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.371 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f07a3d5b980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:06:07.371 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f07a3d5b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:07.371 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f07a3d5b980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:06:07.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.371+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.375+0000 7f07a3d5b980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:06:07.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.375+0000 7f07a3d5b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:07.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.375+0000 7f07a3d5b980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.375+0000 7f07a3d5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.375 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.375+0000 7f06537fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:07.379 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.379+0000 7f07a3d5b980 20 remove_watcher() i=0 2024-03-04T18:06:07.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.379+0000 7f07a3d5b980 2 removed watcher, disabling cache 2024-03-04T18:06:07.381 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.383+0000 7f07a3d5b980 20 remove_watcher() i=1 2024-03-04T18:06:07.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.383+0000 7f07a3d5b980 20 remove_watcher() i=2 2024-03-04T18:06:07.385 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.387+0000 7f07a3d5b980 20 remove_watcher() i=3 2024-03-04T18:06:07.387 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.387+0000 7f07a3d5b980 20 remove_watcher() i=4 2024-03-04T18:06:07.389 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.391+0000 7f07a3d5b980 20 remove_watcher() i=5 2024-03-04T18:06:07.392 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.395+0000 7f07a3d5b980 20 remove_watcher() i=6 2024-03-04T18:06:07.395 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.395+0000 7f07a3d5b980 20 remove_watcher() i=7 2024-03-04T18:06:07.409 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:06:07.409 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:06:07.409 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: a-bucket id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 num_objects: 0 size_kb: 0 num_shards: 3 2024-03-04T18:06:07.410 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:06:07.410 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket layout --bucket a-bucket 2024-03-04T18:06:07.436 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:07.436 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:07.478 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.479+0000 7ff7c8434980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.483+0000 7ff7c8434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.483+0000 7ff7c8434980 20 realm 2024-03-04T18:06:07.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.483+0000 7ff7c8434980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.483+0000 7ff7c8434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.483+0000 7ff7c8434980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:07.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.483+0000 7ff7c8434980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.483+0000 7ff7c8434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.483+0000 7ff7c8434980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.483+0000 7ff7c8434980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:07.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.483+0000 7ff7c8434980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.483+0000 7ff7c8434980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:07.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.487+0000 7ff7c8434980 20 searching for the correct realm 2024-03-04T18:06:07.489 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.491+0000 7ff7c674d640 0 -- 172.21.15.137:0/1142685672 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7ff7c0059d40 msgr2=0x7ff7c00580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:07.706 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.707+0000 7ff7c674d640 0 -- 172.21.15.137:0/1142685672 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x56453d9200c0 msgr2=0x56453d922550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:07.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.715+0000 7ff7c8434980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:07.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.715+0000 7ff7c8434980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:07.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.715+0000 7ff7c8434980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:07.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.715+0000 7ff7c8434980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:07.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.715+0000 7ff7c8434980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:07.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.715+0000 7ff7c8434980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:07.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.715+0000 7ff7c8434980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:07.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:07.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 20 zone default found 2024-03-04T18:06:07.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 4 Realm: () 2024-03-04T18:06:07.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:07.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:07.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:07.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 20 zonegroup default 2024-03-04T18:06:07.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.719+0000 7ff7c8434980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.723+0000 7ff7c8434980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.723+0000 7ff7c8434980 20 started sync module instance, tier type = 2024-03-04T18:06:07.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.723+0000 7ff7c8434980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:07.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.743+0000 7ff7c8434980 20 add_watcher() i=0 2024-03-04T18:06:07.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.743+0000 7ff7c8434980 20 add_watcher() i=1 2024-03-04T18:06:07.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.743+0000 7ff7c8434980 20 add_watcher() i=2 2024-03-04T18:06:07.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.743+0000 7ff7c8434980 20 add_watcher() i=3 2024-03-04T18:06:07.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.743+0000 7ff7c8434980 20 add_watcher() i=4 2024-03-04T18:06:07.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.743+0000 7ff7c8434980 20 add_watcher() i=5 2024-03-04T18:06:07.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.743+0000 7ff7c8434980 20 add_watcher() i=6 2024-03-04T18:06:07.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.747+0000 7ff7c8434980 20 add_watcher() i=7 2024-03-04T18:06:07.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.747+0000 7ff7c8434980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:07.749 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.751+0000 7ff6777fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:07.749 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.751+0000 7ff7c8434980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:07.749 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.751+0000 7ff7c8434980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:07.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.751+0000 7ff7c8434980 5 note: GC not initialized 2024-03-04T18:06:07.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.751+0000 7ff6767fc640 20 reqs_thread_entry: start 2024-03-04T18:06:07.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.751+0000 7ff7c8434980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:07.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.751+0000 7ff7c8434980 20 Filter name: none 2024-03-04T18:06:07.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.751+0000 7ff5e7fff640 20 reqs_thread_entry: start 2024-03-04T18:06:07.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.751+0000 7ff7c8434980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.751+0000 7ff7c8434980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:06:07.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.751+0000 7ff7c8434980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.755+0000 7ff7c8434980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:06:07.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.755+0000 7ff7c8434980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:07.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.755+0000 7ff6777fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:07.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.759+0000 7ff7c8434980 20 remove_watcher() i=0 2024-03-04T18:06:07.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.759+0000 7ff7c8434980 2 removed watcher, disabling cache 2024-03-04T18:06:07.759 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.759+0000 7ff7c8434980 20 remove_watcher() i=1 2024-03-04T18:06:07.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.763+0000 7ff7c8434980 20 remove_watcher() i=2 2024-03-04T18:06:07.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.767+0000 7ff7c8434980 20 remove_watcher() i=3 2024-03-04T18:06:07.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.767+0000 7ff7c8434980 20 remove_watcher() i=4 2024-03-04T18:06:07.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.771+0000 7ff7c8434980 20 remove_watcher() i=5 2024-03-04T18:06:07.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.775+0000 7ff7c8434980 20 remove_watcher() i=6 2024-03-04T18:06:07.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.775+0000 7ff7c8434980 20 remove_watcher() i=7 2024-03-04T18:06:07.791 INFO:tasks.workunit.client.0.smithi137.stderr:rados -p default.rgw.buckets.index setomapval .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.0 --omap-key-file /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_key < /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_val 2024-03-04T18:06:07.817 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:07.817 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:07.837 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket a-bucket 2024-03-04T18:06:07.862 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:07.862 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:07.903 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.903+0000 7f1ee77b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.907+0000 7f1ee77b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.907+0000 7f1ee77b2980 20 realm 2024-03-04T18:06:07.906 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.907+0000 7f1ee77b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.907+0000 7f1ee77b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.907+0000 7f1ee77b2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:07.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.907+0000 7f1ee77b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.907+0000 7f1ee77b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:07.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.907+0000 7f1ee77b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.907+0000 7f1ee77b2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:07.907 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.907+0000 7f1ee77b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:07.908 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.911+0000 7f1ee77b2980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:07.908 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.911+0000 7f1ee77b2980 20 searching for the correct realm 2024-03-04T18:06:07.912 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:07.915+0000 7f1ee4ac9640 0 -- 172.21.15.137:0/4005997547 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f1ed81207d0 msgr2=0x7f1ed8115a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:08.128 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.131+0000 7f1ee4ac9640 0 -- 172.21.15.137:0/4005997547 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x562b5f129440 msgr2=0x562b5f12b830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:08.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.143+0000 7f1ee77b2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:08.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.143+0000 7f1ee77b2980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:08.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.143+0000 7f1ee77b2980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:08.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.143+0000 7f1ee77b2980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:08.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.143+0000 7f1ee77b2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:08.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.143+0000 7f1ee77b2980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:08.144 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.143+0000 7f1ee77b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:08.145 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:08.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 20 zone default found 2024-03-04T18:06:08.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 4 Realm: () 2024-03-04T18:06:08.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:08.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:08.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:08.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 20 zonegroup default 2024-03-04T18:06:08.146 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.147 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.147+0000 7f1ee77b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.151+0000 7f1ee77b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.151+0000 7f1ee77b2980 20 started sync module instance, tier type = 2024-03-04T18:06:08.148 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.151+0000 7f1ee77b2980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:08.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.171+0000 7f1ee77b2980 20 add_watcher() i=0 2024-03-04T18:06:08.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.171+0000 7f1ee77b2980 20 add_watcher() i=1 2024-03-04T18:06:08.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.171+0000 7f1ee77b2980 20 add_watcher() i=2 2024-03-04T18:06:08.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.171+0000 7f1ee77b2980 20 add_watcher() i=3 2024-03-04T18:06:08.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.171+0000 7f1ee77b2980 20 add_watcher() i=4 2024-03-04T18:06:08.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.171+0000 7f1ee77b2980 20 add_watcher() i=5 2024-03-04T18:06:08.170 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.171+0000 7f1ee77b2980 20 add_watcher() i=6 2024-03-04T18:06:08.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.175+0000 7f1ee77b2980 20 add_watcher() i=7 2024-03-04T18:06:08.172 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.175+0000 7f1ee77b2980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:08.176 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.175+0000 7f1d9b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:08.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:08.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:08.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 5 note: GC not initialized 2024-03-04T18:06:08.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1d9a7e4640 20 reqs_thread_entry: start 2024-03-04T18:06:08.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:08.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 Filter name: none 2024-03-04T18:06:08.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1d0bfff640 20 reqs_thread_entry: start 2024-03-04T18:06:08.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.177 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:06:08.178 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:06:08.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:08.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-03-04T18:06:08.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 INFO: main: starting shard=0 2024-03-04T18:06:08.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.0 2024-03-04T18:06:08.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.0 2024-03-04T18:06:08.179 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T18:06:08.180 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 INFO: main: starting shard=1 2024-03-04T18:06:08.180 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.1 2024-03-04T18:06:08.180 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.179+0000 7f1ee77b2980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.1 2024-03-04T18:06:08.181 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.183+0000 7f1ee77b2980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:08.181 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.183+0000 7f1ee77b2980 20 INFO: main: starting shard=2 2024-03-04T18:06:08.181 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.183+0000 7f1ee77b2980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.2 2024-03-04T18:06:08.181 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.183+0000 7f1ee77b2980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.2 2024-03-04T18:06:08.182 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.183+0000 7f1ee77b2980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:08.182 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.183+0000 7f1ee77b2980 20 INFO: main: done 2024-03-04T18:06:08.183 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.183+0000 7f1d9b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:08.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.187+0000 7f1ee77b2980 20 remove_watcher() i=0 2024-03-04T18:06:08.187 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.187+0000 7f1ee77b2980 2 removed watcher, disabling cache 2024-03-04T18:06:08.189 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.191+0000 7f1ee77b2980 20 remove_watcher() i=1 2024-03-04T18:06:08.191 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.191+0000 7f1ee77b2980 20 remove_watcher() i=2 2024-03-04T18:06:08.193 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.195+0000 7f1ee77b2980 20 remove_watcher() i=3 2024-03-04T18:06:08.195 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.195+0000 7f1ee77b2980 20 remove_watcher() i=4 2024-03-04T18:06:08.197 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.199+0000 7f1ee77b2980 20 remove_watcher() i=5 2024-03-04T18:06:08.199 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.199+0000 7f1ee77b2980 20 remove_watcher() i=6 2024-03-04T18:06:08.202 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.203+0000 7f1ee77b2980 20 remove_watcher() i=7 2024-03-04T18:06:08.217 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket a-bucket --num-shards 1 --yes-i-really-mean-it 2024-03-04T18:06:08.247 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:08.247 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:08.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.291+0000 7f35b1906980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.291 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.291+0000 7f35b1906980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.291 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.291+0000 7f35b1906980 20 realm 2024-03-04T18:06:08.291 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.291+0000 7f35b1906980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.292 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.295+0000 7f35b1906980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.292 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.295+0000 7f35b1906980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:08.292 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.295+0000 7f35b1906980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.292 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.295+0000 7f35b1906980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.292 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.295+0000 7f35b1906980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.295+0000 7f35b1906980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:08.293 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.295+0000 7f35b1906980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.295+0000 7f35b1906980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:08.294 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.295+0000 7f35b1906980 20 searching for the correct realm 2024-03-04T18:06:08.297 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.299+0000 7f35af41e640 0 -- 172.21.15.137:0/2179380831 >> [v2:172.21.15.137:6824/1039931256,v1:172.21.15.137:6825/1039931256] conn(0x55d4eadfeed0 msgr2=0x55d4eae012e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:08.303 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.303+0000 7f35af41e640 0 -- 172.21.15.137:0/2179380831 >> [v2:172.21.15.173:6805/879938785,v1:172.21.15.173:6808/879938785] conn(0x7f35a00f9980 msgr2=0x7f35a01088d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:08.521 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:08.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:08.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:08.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:08.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:08.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:08.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.522 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:08.523 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.524 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:08.524 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 zone default found 2024-03-04T18:06:08.524 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 4 Realm: () 2024-03-04T18:06:08.524 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:08.524 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:08.524 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:08.524 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 zonegroup default 2024-03-04T18:06:08.524 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.523+0000 7f35b1906980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.527+0000 7f35b1906980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.527+0000 7f35b1906980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.527+0000 7f35b1906980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.525 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.527+0000 7f35b1906980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.526 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.527+0000 7f35b1906980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.526 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.527+0000 7f35b1906980 20 started sync module instance, tier type = 2024-03-04T18:06:08.526 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.527+0000 7f35b1906980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:08.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.551+0000 7f35b1906980 20 add_watcher() i=0 2024-03-04T18:06:08.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.551+0000 7f35b1906980 20 add_watcher() i=1 2024-03-04T18:06:08.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.551+0000 7f35b1906980 20 add_watcher() i=2 2024-03-04T18:06:08.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.551+0000 7f35b1906980 20 add_watcher() i=3 2024-03-04T18:06:08.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.551+0000 7f35b1906980 20 add_watcher() i=4 2024-03-04T18:06:08.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.551+0000 7f35b1906980 20 add_watcher() i=5 2024-03-04T18:06:08.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.551+0000 7f35b1906980 20 add_watcher() i=6 2024-03-04T18:06:08.551 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.551+0000 7f35b1906980 20 add_watcher() i=7 2024-03-04T18:06:08.552 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.551+0000 7f35b1906980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:08.556 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f3465fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:08.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f35b1906980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:08.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f35b1906980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:08.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f35b1906980 5 note: GC not initialized 2024-03-04T18:06:08.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f3464fe1640 20 reqs_thread_entry: start 2024-03-04T18:06:08.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f35b1906980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:08.558 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f35b1906980 20 Filter name: none 2024-03-04T18:06:08.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f33de7fc640 20 reqs_thread_entry: start 2024-03-04T18:06:08.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f35b1906980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-03-04T18:06:08.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f35b1906980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f35b1906980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:06:08.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f35b1906980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-03-04T18:06:08.559 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f35b1906980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-03-04T18:06:08.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f35b1906980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 : miss 2024-03-04T18:06:08.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.559+0000 7f35b1906980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.563+0000 7f35b1906980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:06:08.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.563+0000 7f35b1906980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:06:08.561 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.563+0000 7f35b1906980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:06:08.561 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.563+0000 7f35b1906980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:08.561 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.563+0000 7f35b1906980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:08.561 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.563+0000 7f35b1906980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 2024-03-04T18:06:08.568 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.571+0000 7f35b1906980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:06:08.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.571+0000 7f35b1906980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:06:08.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.571+0000 7f35b1906980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:08.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.571+0000 7f35b1906980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18, ofs0, ns] 2024-03-04T18:06:08.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.571+0000 7f34927fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627829 cookie 94372962029904 notifier 6725 bl.length()=861 2024-03-04T18:06:08.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.571+0000 7f34927fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:06:08.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.571+0000 7f34927fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:06:08.570 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.571+0000 7f34927fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:08.573 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.575+0000 7f35b1906980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): orig_sources= new_sources= 2024-03-04T18:06:08.573 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.575+0000 7f35b1906980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): potential sources added=[] removed=[] 2024-03-04T18:06:08.574 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.575+0000 7f35b1906980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): orig_dests= new_dests= 2024-03-04T18:06:08.574 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.575+0000 7f35b1906980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): potential dests added=[] removed=[] 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.579+0000 7f35b1906980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.3.0 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.579+0000 7f35b1906980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.3.0 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.579+0000 7f35b1906980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.0 2024-03-04T18:06:08.577 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.579+0000 7f35b1906980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.0 2024-03-04T18:06:08.578 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.579+0000 7f35b1906980 10 Dropping entry with empty name, idx=�1001_04/57/0457f727ec113e418d5b16d206b200ed068c0533554883ce811df7c932a3df68/2018_12_11/2889999/3386469/metadata.gz 2024-03-04T18:06:08.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.579+0000 7f35b1906980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.1 2024-03-04T18:06:08.579 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.579+0000 7f35b1906980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.1 2024-03-04T18:06:08.580 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.583+0000 7f35b1906980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.2 2024-03-04T18:06:08.580 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.583+0000 7f35b1906980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.2.2 2024-03-04T18:06:08.581 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.583+0000 7f35b1906980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 : hit (requested=0x17, cached=0x17) 2024-03-04T18:06:08.581 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.583+0000 7f35b1906980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 2024-03-04T18:06:08.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.587+0000 7f35b1906980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:06:08.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.587+0000 7f35b1906980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:06:08.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.587+0000 7f35b1906980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:08.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.587+0000 7f35b1906980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18, ofs0, ns] 2024-03-04T18:06:08.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.587+0000 7f3492ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627830 cookie 94372962029904 notifier 6725 bl.length()=829 2024-03-04T18:06:08.585 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.587+0000 7f3492ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 info.flags=0x17 2024-03-04T18:06:08.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.587+0000 7f3492ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18 to cache LRU end 2024-03-04T18:06:08.586 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.587+0000 7f3492ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:08.588 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.587+0000 7f35b1906980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): orig_sources= new_sources= 2024-03-04T18:06:08.588 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.587+0000 7f35b1906980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): potential sources added=[] removed=[] 2024-03-04T18:06:08.588 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.587+0000 7f35b1906980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): orig_dests= new_dests= 2024-03-04T18:06:08.588 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.587+0000 7f35b1906980 20 handle_bi_update(): bucket=:a-bucket[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18]): potential dests added=[] removed=[] 2024-03-04T18:06:08.591 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.591+0000 7f35b1906980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-03-04T18:06:08.595 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.595+0000 7f3465fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:08.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.599+0000 7f35b1906980 20 remove_watcher() i=0 2024-03-04T18:06:08.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.599+0000 7f35b1906980 2 removed watcher, disabling cache 2024-03-04T18:06:08.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.603+0000 7f35b1906980 20 remove_watcher() i=1 2024-03-04T18:06:08.604 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.607+0000 7f35b1906980 20 remove_watcher() i=2 2024-03-04T18:06:08.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.607+0000 7f35b1906980 20 remove_watcher() i=3 2024-03-04T18:06:08.608 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.611+0000 7f35b1906980 20 remove_watcher() i=4 2024-03-04T18:06:08.610 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.611+0000 7f35b1906980 20 remove_watcher() i=5 2024-03-04T18:06:08.613 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.615+0000 7f35b1906980 20 remove_watcher() i=6 2024-03-04T18:06:08.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.619+0000 7f35b1906980 20 remove_watcher() i=7 2024-03-04T18:06:08.633 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket a-bucket 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:08.663 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:08.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.707+0000 7f3e387db980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.707+0000 7f3e387db980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.707+0000 7f3e387db980 20 realm 2024-03-04T18:06:08.707 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.707+0000 7f3e387db980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.707+0000 7f3e387db980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.707+0000 7f3e387db980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:08.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.707+0000 7f3e387db980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.711+0000 7f3e387db980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.708 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.711+0000 7f3e387db980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.711+0000 7f3e387db980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:08.709 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.711+0000 7f3e387db980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.711+0000 7f3e387db980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:08.710 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.711+0000 7f3e387db980 20 searching for the correct realm 2024-03-04T18:06:08.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.715+0000 7f3e36af4640 0 -- 172.21.15.137:0/2815890009 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f3e3005ffb0 msgr2=0x7f3e3006f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:08.930 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.931+0000 7f3e36af4640 0 -- 172.21.15.137:0/2815890009 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x563e47944250 msgr2=0x563e479466e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:08.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.939+0000 7f3e387db980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:08.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.939+0000 7f3e387db980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:08.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.939+0000 7f3e387db980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:08.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.939+0000 7f3e387db980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:08.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.939+0000 7f3e387db980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:08.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.939+0000 7f3e387db980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:08.939 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.939+0000 7f3e387db980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.939+0000 7f3e387db980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.939+0000 7f3e387db980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.940 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:08.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:08.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 20 zone default found 2024-03-04T18:06:08.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 4 Realm: () 2024-03-04T18:06:08.941 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:08.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:08.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:08.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 20 zonegroup default 2024-03-04T18:06:08.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.942 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.943 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:08.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 20 started sync module instance, tier type = 2024-03-04T18:06:08.944 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.943+0000 7f3e387db980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:08.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.967+0000 7f3e387db980 20 add_watcher() i=0 2024-03-04T18:06:08.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.967+0000 7f3e387db980 20 add_watcher() i=1 2024-03-04T18:06:08.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.967+0000 7f3e387db980 20 add_watcher() i=2 2024-03-04T18:06:08.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.967+0000 7f3e387db980 20 add_watcher() i=3 2024-03-04T18:06:08.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.967+0000 7f3e387db980 20 add_watcher() i=4 2024-03-04T18:06:08.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.967+0000 7f3e387db980 20 add_watcher() i=5 2024-03-04T18:06:08.968 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.967+0000 7f3e387db980 20 add_watcher() i=6 2024-03-04T18:06:08.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.971+0000 7f3e387db980 20 add_watcher() i=7 2024-03-04T18:06:08.970 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.971+0000 7f3e387db980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:08.974 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.975+0000 7f3cecfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:08.975 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.975+0000 7f3e387db980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:08.975 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.975+0000 7f3e387db980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:08.975 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.975+0000 7f3e387db980 5 note: GC not initialized 2024-03-04T18:06:08.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.975+0000 7f3c677fe640 20 reqs_thread_entry: start 2024-03-04T18:06:08.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.975+0000 7f3e387db980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:08.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.979+0000 7f3e387db980 20 Filter name: none 2024-03-04T18:06:08.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.979+0000 7f3c657fa640 20 reqs_thread_entry: start 2024-03-04T18:06:08.976 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.979+0000 7f3e387db980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.979+0000 7f3e387db980 20 rados_obj.operate() r=0 bl.length=200 2024-03-04T18:06:08.977 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.979+0000 7f3e387db980 20 rados->read ofs=0 len=0 2024-03-04T18:06:08.978 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.979+0000 7f3e387db980 20 rados_obj.operate() r=0 bl.length=372 2024-03-04T18:06:08.978 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.979+0000 7f3e387db980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:08.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.979+0000 7f3e387db980 20 INFO: main: max_entries=1000, index=gen=3, max_shards=1 2024-03-04T18:06:08.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.979+0000 7f3e387db980 20 INFO: main: starting shard=0 2024-03-04T18:06:08.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.979+0000 7f3e387db980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.3.0 2024-03-04T18:06:08.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.979+0000 7f3e387db980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18.3.0 2024-03-04T18:06:08.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.979+0000 7f3e387db980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:08.979 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.979+0000 7f3e387db980 20 INFO: main: done 2024-03-04T18:06:08.980 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.983+0000 7f3cecfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:08.985 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.987+0000 7f3e387db980 20 remove_watcher() i=0 2024-03-04T18:06:08.985 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.987+0000 7f3e387db980 2 removed watcher, disabling cache 2024-03-04T18:06:08.987 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.987+0000 7f3e387db980 20 remove_watcher() i=1 2024-03-04T18:06:08.989 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.991+0000 7f3e387db980 20 remove_watcher() i=2 2024-03-04T18:06:08.991 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.991+0000 7f3e387db980 20 remove_watcher() i=3 2024-03-04T18:06:08.993 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.995+0000 7f3e387db980 20 remove_watcher() i=4 2024-03-04T18:06:08.995 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.995+0000 7f3e387db980 20 remove_watcher() i=5 2024-03-04T18:06:08.998 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:08.999+0000 7f3e387db980 20 remove_watcher() i=6 2024-03-04T18:06:09.001 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.003+0000 7f3e387db980 20 remove_watcher() i=7 2024-03-04T18:06:09.017 INFO:tasks.workunit.client.0.smithi137.stderr: test: PUT succeeds during reshard 2024-03-04T18:06:09.018 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket myver 2024-03-04T18:06:09.044 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:09.044 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:09.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.087+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.091+0000 7fd6dcd15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.091+0000 7fd6dcd15980 20 realm 2024-03-04T18:06:09.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.091+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.091+0000 7fd6dcd15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.091+0000 7fd6dcd15980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:09.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.091+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.091+0000 7fd6dcd15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.089 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.091+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.091+0000 7fd6dcd15980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:09.090 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.091+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.091 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.091+0000 7fd6dcd15980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:09.091 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.091+0000 7fd6dcd15980 20 searching for the correct realm 2024-03-04T18:06:09.099 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.099+0000 7fd6db02e640 0 -- 172.21.15.137:0/3010136506 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fd6d40094b0 msgr2=0x7fd6d4009be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:09.314 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.315+0000 7fd6db02e640 0 -- 172.21.15.137:0/3010136506 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x5607d7fa0880 msgr2=0x5607d7fa2d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:09.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.323+0000 7fd6dcd15980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:09.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.323+0000 7fd6dcd15980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:09.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.323+0000 7fd6dcd15980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:09.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.323+0000 7fd6dcd15980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:09.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.323+0000 7fd6dcd15980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:09.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.323+0000 7fd6dcd15980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:09.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.323+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.323+0000 7fd6dcd15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.323+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.324 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:09.324 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.324 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:09.324 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 20 zone default found 2024-03-04T18:06:09.324 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 4 Realm: () 2024-03-04T18:06:09.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:09.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:09.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:09.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 20 zonegroup default 2024-03-04T18:06:09.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.325 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.326 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.326 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.326 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.326 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 20 started sync module instance, tier type = 2024-03-04T18:06:09.326 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.327+0000 7fd6dcd15980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:09.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.351+0000 7fd6dcd15980 20 add_watcher() i=0 2024-03-04T18:06:09.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.351+0000 7fd6dcd15980 20 add_watcher() i=1 2024-03-04T18:06:09.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.351+0000 7fd6dcd15980 20 add_watcher() i=2 2024-03-04T18:06:09.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.351+0000 7fd6dcd15980 20 add_watcher() i=3 2024-03-04T18:06:09.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.351+0000 7fd6dcd15980 20 add_watcher() i=4 2024-03-04T18:06:09.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.351+0000 7fd6dcd15980 20 add_watcher() i=5 2024-03-04T18:06:09.349 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.351+0000 7fd6dcd15980 20 add_watcher() i=6 2024-03-04T18:06:09.351 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.351+0000 7fd6dcd15980 20 add_watcher() i=7 2024-03-04T18:06:09.352 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.351+0000 7fd6dcd15980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:09.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.355+0000 7fd5917e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:09.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.359+0000 7fd6dcd15980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:09.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.359+0000 7fd6dcd15980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:09.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.359+0000 7fd6dcd15980 5 note: GC not initialized 2024-03-04T18:06:09.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.359+0000 7fd50bfff640 20 reqs_thread_entry: start 2024-03-04T18:06:09.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.359+0000 7fd6dcd15980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:09.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.359+0000 7fd6dcd15980 20 Filter name: none 2024-03-04T18:06:09.358 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.359+0000 7fd509ffb640 20 reqs_thread_entry: start 2024-03-04T18:06:09.358 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.359+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.358 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.359+0000 7fd6dcd15980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:09.358 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.359+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.360 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.359+0000 7fd6dcd15980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:09.360 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.359+0000 7fd6dcd15980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:09.360 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.363+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.361 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.363+0000 7fd6dcd15980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:09.361 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.363+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.362 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.363+0000 7fd6dcd15980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:09.362 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.363+0000 7fd6dcd15980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:09.363 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.363+0000 7fd6dcd15980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.363 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.363+0000 7fd6dcd15980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.365 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.367+0000 7fd5917e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:09.370 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.371+0000 7fd6dcd15980 20 remove_watcher() i=0 2024-03-04T18:06:09.370 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.371+0000 7fd6dcd15980 2 removed watcher, disabling cache 2024-03-04T18:06:09.372 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.375+0000 7fd6dcd15980 20 remove_watcher() i=1 2024-03-04T18:06:09.374 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.375+0000 7fd6dcd15980 20 remove_watcher() i=2 2024-03-04T18:06:09.376 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.379+0000 7fd6dcd15980 20 remove_watcher() i=3 2024-03-04T18:06:09.378 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.379+0000 7fd6dcd15980 20 remove_watcher() i=4 2024-03-04T18:06:09.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.383+0000 7fd6dcd15980 20 remove_watcher() i=5 2024-03-04T18:06:09.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.383+0000 7fd6dcd15980 20 remove_watcher() i=6 2024-03-04T18:06:09.386 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.387+0000 7fd6dcd15980 20 remove_watcher() i=7 2024-03-04T18:06:09.403 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:06:09.403 INFO:tasks.workunit.client.0.smithi137.stderr:BUCKET_STATS: 2024-03-04T18:06:09.403 INFO:tasks.workunit.client.0.smithi137.stderr:bucket: myver id: f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 num_objects: 0 size_kb: 0 num_shards: 2 2024-03-04T18:06:09.403 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:06:09.403 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin --inject-delay-at=do_reshard --inject-delay-ms=5000 bucket reshard --bucket myver --num-shards 3 2024-03-04T18:06:09.438 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:09.438 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:09.478 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.479+0000 7f85f0325980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.480 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.483+0000 7f85f0325980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.483+0000 7f85f0325980 20 realm 2024-03-04T18:06:09.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.483+0000 7f85f0325980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.483+0000 7f85f0325980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.483+0000 7f85f0325980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:09.481 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.483+0000 7f85f0325980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.483+0000 7f85f0325980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.483+0000 7f85f0325980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.483+0000 7f85f0325980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:09.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.483+0000 7f85f0325980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.483+0000 7f85f0325980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:09.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.483+0000 7f85f0325980 20 searching for the correct realm 2024-03-04T18:06:09.488 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.491+0000 7f85ee63e640 0 -- 172.21.15.137:0/774117178 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f85e0054b90 msgr2=0x7f85e00598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:09.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.707+0000 7f85ee63e640 0 -- 172.21.15.137:0/774117178 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x565344aee9e0 msgr2=0x565344af0ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:09.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.715+0000 7f85f0325980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:09.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.715+0000 7f85f0325980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:09.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.715+0000 7f85f0325980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:09.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.715+0000 7f85f0325980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:09.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.715+0000 7f85f0325980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:09.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.715+0000 7f85f0325980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:09.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.715+0000 7f85f0325980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.715+0000 7f85f0325980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.715+0000 7f85f0325980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.719+0000 7f85f0325980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:09.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.719+0000 7f85f0325980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.719+0000 7f85f0325980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:09.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.719+0000 7f85f0325980 20 zone default found 2024-03-04T18:06:09.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.719+0000 7f85f0325980 4 Realm: () 2024-03-04T18:06:09.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.719+0000 7f85f0325980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:09.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.719+0000 7f85f0325980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:09.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.719+0000 7f85f0325980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:09.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.719+0000 7f85f0325980 20 zonegroup default 2024-03-04T18:06:09.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.719+0000 7f85f0325980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.719+0000 7f85f0325980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.719+0000 7f85f0325980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.719+0000 7f85f0325980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.719+0000 7f85f0325980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.723+0000 7f85f0325980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:09.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.723+0000 7f85f0325980 20 started sync module instance, tier type = 2024-03-04T18:06:09.720 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.723+0000 7f85f0325980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:09.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.743+0000 7f85f0325980 20 add_watcher() i=0 2024-03-04T18:06:09.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.743+0000 7f85f0325980 20 add_watcher() i=1 2024-03-04T18:06:09.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.743+0000 7f85f0325980 20 add_watcher() i=2 2024-03-04T18:06:09.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.743+0000 7f85f0325980 20 add_watcher() i=3 2024-03-04T18:06:09.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.743+0000 7f85f0325980 20 add_watcher() i=4 2024-03-04T18:06:09.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.743+0000 7f85f0325980 20 add_watcher() i=5 2024-03-04T18:06:09.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.743+0000 7f85f0325980 20 add_watcher() i=6 2024-03-04T18:06:09.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.747+0000 7f85f0325980 20 add_watcher() i=7 2024-03-04T18:06:09.745 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.747+0000 7f85f0325980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:09.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.751+0000 7f849f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:09.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.751+0000 7f85f0325980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:09.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.751+0000 7f85f0325980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:09.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.751+0000 7f85f0325980 5 note: GC not initialized 2024-03-04T18:06:09.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.751+0000 7f849e7fc640 20 reqs_thread_entry: start 2024-03-04T18:06:09.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.751+0000 7f85f0325980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:09.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.751+0000 7f85f0325980 20 Filter name: none 2024-03-04T18:06:09.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.755+0000 7f840ffff640 20 reqs_thread_entry: start 2024-03-04T18:06:09.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.755+0000 7f85f0325980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-03-04T18:06:09.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.755+0000 7f85f0325980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.755+0000 7f85f0325980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:09.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.755+0000 7f85f0325980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-03-04T18:06:09.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.755+0000 7f85f0325980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-03-04T18:06:09.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.755+0000 7f85f0325980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 : miss 2024-03-04T18:06:09.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.755+0000 7f85f0325980 20 rados->read ofs=0 len=0 2024-03-04T18:06:09.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.755+0000 7f85f0325980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:09.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.755+0000 7f85f0325980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:09.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.755+0000 7f85f0325980 10 adding default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:09.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.755+0000 7f85f0325980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:09.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.755+0000 7f85f0325980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:09.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.755+0000 7f85f0325980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 2024-03-04T18:06:09.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.763+0000 7f85f0325980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:09.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.763+0000 7f85f0325980 10 moving default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:09.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.763+0000 7f85f0325980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:09.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.763+0000 7f85f0325980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19, ofs0, ns] 2024-03-04T18:06:09.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.767+0000 7f84d0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627855 cookie 94915634593936 notifier 6776 bl.length()=855 2024-03-04T18:06:09.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.767+0000 7f84d0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:09.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.767+0000 7f84d0ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:09.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.767+0000 7f84d0ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:09.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.771+0000 7f85f0325980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): orig_sources= new_sources= 2024-03-04T18:06:09.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.771+0000 7f85f0325980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): potential sources added=[] removed=[] 2024-03-04T18:06:09.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.771+0000 7f85f0325980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): orig_dests= new_dests= 2024-03-04T18:06:09.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.771+0000 7f85f0325980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): potential dests added=[] removed=[] 2024-03-04T18:06:09.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:09.771+0000 7f85f0325980 -1 Injecting delay=5.000000000s at location=do_reshard 2024-03-04T18:06:10.408 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:10.408 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:14.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.771+0000 7f85f0325980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.0 2024-03-04T18:06:14.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.771+0000 7f85f0325980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.0 2024-03-04T18:06:14.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.771+0000 7f85f0325980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.1 2024-03-04T18:06:14.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.771+0000 7f85f0325980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.1 2024-03-04T18:06:14.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.771+0000 7f85f0325980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.2 2024-03-04T18:06:14.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.771+0000 7f85f0325980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.2 2024-03-04T18:06:14.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.771+0000 7f85f0325980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.1.0 2024-03-04T18:06:14.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.771+0000 7f85f0325980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.1.0 2024-03-04T18:06:14.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.775+0000 7f85f0325980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.1.1 2024-03-04T18:06:14.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.775+0000 7f85f0325980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.1.1 2024-03-04T18:06:14.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.775+0000 7f85f0325980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 : hit (requested=0x17, cached=0x17) 2024-03-04T18:06:14.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.775+0000 7f85f0325980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 2024-03-04T18:06:14.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.779+0000 7f85f0325980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:14.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.779+0000 7f85f0325980 10 moving default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:14.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.779+0000 7f85f0325980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:14.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.779+0000 7f85f0325980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19, ofs0, ns] 2024-03-04T18:06:14.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.779+0000 7f84d0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627856 cookie 94915634593936 notifier 6776 bl.length()=823 2024-03-04T18:06:14.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.779+0000 7f84d0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:14.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.779+0000 7f84d0ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:14.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.779+0000 7f84d0ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:14.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.783+0000 7f85f0325980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): orig_sources= new_sources= 2024-03-04T18:06:14.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.783+0000 7f85f0325980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): potential sources added=[] removed=[] 2024-03-04T18:06:14.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.783+0000 7f85f0325980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): orig_dests= new_dests= 2024-03-04T18:06:14.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.783+0000 7f85f0325980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): potential dests added=[] removed=[] 2024-03-04T18:06:14.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.787+0000 7f85f0325980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-03-04T18:06:14.791 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.791+0000 7f849f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:14.797 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.799+0000 7f85f0325980 20 remove_watcher() i=0 2024-03-04T18:06:14.797 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.799+0000 7f85f0325980 2 removed watcher, disabling cache 2024-03-04T18:06:14.799 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.799+0000 7f85f0325980 20 remove_watcher() i=1 2024-03-04T18:06:14.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.803+0000 7f85f0325980 20 remove_watcher() i=2 2024-03-04T18:06:14.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.807+0000 7f85f0325980 20 remove_watcher() i=3 2024-03-04T18:06:14.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.807+0000 7f85f0325980 20 remove_watcher() i=4 2024-03-04T18:06:14.809 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.811+0000 7f85f0325980 20 remove_watcher() i=5 2024-03-04T18:06:14.812 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.815+0000 7f85f0325980 20 remove_watcher() i=6 2024-03-04T18:06:14.816 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:14.819+0000 7f85f0325980 20 remove_watcher() i=7 2024-03-04T18:06:15.597 INFO:tasks.workunit.client.0.smithi137.stderr:put object successful 2024-03-04T18:06:15.597 INFO:tasks.workunit.client.0.smithi137.stderr:TEST: check that bucket stats are correct after reshard with unlinked entries 2024-03-04T18:06:15.597 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-04T18:06:15.602 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:15.602 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:15.706 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:15.706 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:15.855 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:15.855 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:15.991 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:15.991 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:16.055 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:16.055 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:16.178 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:16.178 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:16.244 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:16.245 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:16.376 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:16.376 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:16.463 INFO:tasks.workunit.client.0.smithi137.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-03-04T18:06:16.880 INFO:tasks.workunit.client.0.smithi137.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-03-04T18:06:18.287 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:18.287 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:18.425 INFO:tasks.workunit.client.0.smithi137.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T18:06:18.426 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket myver --object x 2024-03-04T18:06:18.460 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:18.460 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:18.503 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.503+0000 7fe5f9056980 20 rados->read ofs=0 len=0 2024-03-04T18:06:18.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.507+0000 7fe5f9056980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:18.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.507+0000 7fe5f9056980 20 realm 2024-03-04T18:06:18.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.507+0000 7fe5f9056980 20 rados->read ofs=0 len=0 2024-03-04T18:06:18.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.507+0000 7fe5f9056980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:18.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.507+0000 7fe5f9056980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:18.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.507+0000 7fe5f9056980 20 rados->read ofs=0 len=0 2024-03-04T18:06:18.507 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.507+0000 7fe5f9056980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:18.507 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.507+0000 7fe5f9056980 20 rados->read ofs=0 len=0 2024-03-04T18:06:18.507 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.507+0000 7fe5f9056980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:18.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.507+0000 7fe5f9056980 20 rados->read ofs=0 len=0 2024-03-04T18:06:18.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.511+0000 7fe5f9056980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:18.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.511+0000 7fe5f9056980 20 searching for the correct realm 2024-03-04T18:06:18.514 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.515+0000 7fe5f736f640 0 -- 172.21.15.137:0/4032747649 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fe5f00a4040 msgr2=0x7fe5f0119000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:18.728 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.731+0000 7fe5f736f640 0 -- 172.21.15.137:0/4032747649 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x5621149199f0 msgr2=0x56211491be90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:18.737 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.739+0000 7fe5f9056980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:18.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.739+0000 7fe5f9056980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:18.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.739+0000 7fe5f9056980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:18.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.739+0000 7fe5f9056980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:18.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.739+0000 7fe5f9056980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:18.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.739+0000 7fe5f9056980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:18.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.739+0000 7fe5f9056980 20 rados->read ofs=0 len=0 2024-03-04T18:06:18.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.739+0000 7fe5f9056980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:18.738 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.739+0000 7fe5f9056980 20 rados->read ofs=0 len=0 2024-03-04T18:06:18.739 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.739+0000 7fe5f9056980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:18.739 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.739+0000 7fe5f9056980 20 rados->read ofs=0 len=0 2024-03-04T18:06:18.740 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:18.740 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 20 zone default found 2024-03-04T18:06:18.740 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 4 Realm: () 2024-03-04T18:06:18.740 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:18.740 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:18.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:18.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 20 zonegroup default 2024-03-04T18:06:18.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 20 rados->read ofs=0 len=0 2024-03-04T18:06:18.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:18.741 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 20 rados->read ofs=0 len=0 2024-03-04T18:06:18.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:18.742 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 20 rados->read ofs=0 len=0 2024-03-04T18:06:18.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:18.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 20 started sync module instance, tier type = 2024-03-04T18:06:18.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.743+0000 7fe5f9056980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:18.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.767+0000 7fe5f9056980 20 add_watcher() i=0 2024-03-04T18:06:18.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.767+0000 7fe5f9056980 20 add_watcher() i=1 2024-03-04T18:06:18.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.767+0000 7fe5f9056980 20 add_watcher() i=2 2024-03-04T18:06:18.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.767+0000 7fe5f9056980 20 add_watcher() i=3 2024-03-04T18:06:18.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.767+0000 7fe5f9056980 20 add_watcher() i=4 2024-03-04T18:06:18.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.767+0000 7fe5f9056980 20 add_watcher() i=5 2024-03-04T18:06:18.766 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.767+0000 7fe5f9056980 20 add_watcher() i=6 2024-03-04T18:06:18.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.771+0000 7fe5f9056980 20 add_watcher() i=7 2024-03-04T18:06:18.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.771+0000 7fe5f9056980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:18.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.775+0000 7fe4ad7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:18.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.775+0000 7fe5f9056980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:18.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.775+0000 7fe5f9056980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:18.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.775+0000 7fe5f9056980 5 note: GC not initialized 2024-03-04T18:06:18.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.775+0000 7fe427fff640 20 reqs_thread_entry: start 2024-03-04T18:06:18.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.775+0000 7fe5f9056980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:18.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.775+0000 7fe5f9056980 20 Filter name: none 2024-03-04T18:06:18.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.775+0000 7fe425ffb640 20 reqs_thread_entry: start 2024-03-04T18:06:18.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.775+0000 7fe5f9056980 20 rados->read ofs=0 len=0 2024-03-04T18:06:18.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.779+0000 7fe5f9056980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:18.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.779+0000 7fe5f9056980 20 rados->read ofs=0 len=0 2024-03-04T18:06:18.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.779+0000 7fe5f9056980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:18.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.779+0000 7fe5f9056980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:18.778 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.779+0000 7fe5f9056980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-03-04T18:06:18.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.779+0000 7fe5f9056980 20 INFO: main: starting shard=0 2024-03-04T18:06:18.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.779+0000 7fe5f9056980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.0 2024-03-04T18:06:18.779 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.779+0000 7fe5f9056980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.0 2024-03-04T18:06:18.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.779+0000 7fe5f9056980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T18:06:18.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.783+0000 7fe5f9056980 20 INFO: main: starting shard=1 2024-03-04T18:06:18.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.783+0000 7fe5f9056980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.1 2024-03-04T18:06:18.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.783+0000 7fe5f9056980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.1 2024-03-04T18:06:18.781 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.783+0000 7fe5f9056980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:18.781 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.783+0000 7fe5f9056980 20 INFO: main: starting shard=2 2024-03-04T18:06:18.781 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.783+0000 7fe5f9056980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.2 2024-03-04T18:06:18.781 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.783+0000 7fe5f9056980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.2 2024-03-04T18:06:18.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.783+0000 7fe5f9056980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:18.782 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.783+0000 7fe5f9056980 20 INFO: main: done 2024-03-04T18:06:18.783 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.783+0000 7fe4ad7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:18.787 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.787+0000 7fe5f9056980 20 remove_watcher() i=0 2024-03-04T18:06:18.788 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.787+0000 7fe5f9056980 2 removed watcher, disabling cache 2024-03-04T18:06:18.790 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.791+0000 7fe5f9056980 20 remove_watcher() i=1 2024-03-04T18:06:18.792 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.795+0000 7fe5f9056980 20 remove_watcher() i=2 2024-03-04T18:06:18.794 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.795+0000 7fe5f9056980 20 remove_watcher() i=3 2024-03-04T18:06:18.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.799+0000 7fe5f9056980 20 remove_watcher() i=4 2024-03-04T18:06:18.798 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.799+0000 7fe5f9056980 20 remove_watcher() i=5 2024-03-04T18:06:18.801 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.803+0000 7fe5f9056980 20 remove_watcher() i=6 2024-03-04T18:06:18.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:18.807+0000 7fe5f9056980 20 remove_watcher() i=7 2024-03-04T18:06:18.821 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:18.821 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:18.847 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:18.847 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:18.985 INFO:tasks.workunit.client.0.smithi137.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T18:06:18.985 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket myver --object y 2024-03-04T18:06:19.020 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:19.020 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:19.265 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.267+0000 7f7cf40f0640 0 --2- 172.21.15.137:0/621425213 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x55a331adb960 0x55a331aafa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-03-04T18:06:19.269 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.271+0000 7f7cf65d8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.271 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.271+0000 7f7cf65d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:19.272 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.271+0000 7f7cf65d8980 20 realm 2024-03-04T18:06:19.272 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.271+0000 7f7cf65d8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.272 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.275+0000 7f7cf65d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:19.272 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.275+0000 7f7cf65d8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:19.272 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.275+0000 7f7cf65d8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.273 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.275+0000 7f7cf65d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:19.273 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.275+0000 7f7cf65d8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.273 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.275+0000 7f7cf65d8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:19.274 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.275+0000 7f7cf65d8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.275 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.275+0000 7f7cf65d8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:19.275 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.275+0000 7f7cf65d8980 20 searching for the correct realm 2024-03-04T18:06:19.279 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.279+0000 7f7cf40f0640 0 -- 172.21.15.137:0/621425213 >> [v2:172.21.15.137:6824/1039931256,v1:172.21.15.137:6825/1039931256] conn(0x55a331af80a0 msgr2=0x55a331afa490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:19.284 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.287+0000 7f7cf40f0640 0 -- 172.21.15.137:0/621425213 >> [v2:172.21.15.173:6805/879938785,v1:172.21.15.173:6808/879938785] conn(0x7f7ce4024070 msgr2=0x7f7ce4086040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:19.505 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.507+0000 7f7cf65d8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:19.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.507+0000 7f7cf65d8980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:19.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.507+0000 7f7cf65d8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:19.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.507+0000 7f7cf65d8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:19.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.507+0000 7f7cf65d8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:19.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.507+0000 7f7cf65d8980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:19.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.507+0000 7f7cf65d8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.507+0000 7f7cf65d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:19.506 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.507+0000 7f7cf65d8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.507 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.507+0000 7f7cf65d8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:19.507 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.507+0000 7f7cf65d8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:19.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 20 zone default found 2024-03-04T18:06:19.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 4 Realm: () 2024-03-04T18:06:19.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:19.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:19.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:19.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 20 zonegroup default 2024-03-04T18:06:19.508 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.509 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:19.509 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.509 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:19.509 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:19.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 20 started sync module instance, tier type = 2024-03-04T18:06:19.510 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.511+0000 7f7cf65d8980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:19.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.535+0000 7f7cf65d8980 20 add_watcher() i=0 2024-03-04T18:06:19.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.535+0000 7f7cf65d8980 20 add_watcher() i=1 2024-03-04T18:06:19.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.535+0000 7f7cf65d8980 20 add_watcher() i=2 2024-03-04T18:06:19.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.535+0000 7f7cf65d8980 20 add_watcher() i=3 2024-03-04T18:06:19.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.535+0000 7f7cf65d8980 20 add_watcher() i=4 2024-03-04T18:06:19.534 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.535+0000 7f7cf65d8980 20 add_watcher() i=5 2024-03-04T18:06:19.535 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.535+0000 7f7cf65d8980 20 add_watcher() i=6 2024-03-04T18:06:19.536 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.539+0000 7f7cf65d8980 20 add_watcher() i=7 2024-03-04T18:06:19.536 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.539+0000 7f7cf65d8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:19.541 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.543+0000 7f7baafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:19.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.543+0000 7f7cf65d8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:19.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.543+0000 7f7cf65d8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:19.542 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.543+0000 7f7cf65d8980 5 note: GC not initialized 2024-03-04T18:06:19.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.543+0000 7f7ba9fdb640 20 reqs_thread_entry: start 2024-03-04T18:06:19.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.543+0000 7f7cf65d8980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:19.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.543+0000 7f7cf65d8980 20 Filter name: none 2024-03-04T18:06:19.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.543+0000 7f7b1b7fe640 20 reqs_thread_entry: start 2024-03-04T18:06:19.543 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.543+0000 7f7cf65d8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.544 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:19.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:19.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:19.545 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-03-04T18:06:19.546 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 20 INFO: main: starting shard=0 2024-03-04T18:06:19.546 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.0 2024-03-04T18:06:19.546 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.0 2024-03-04T18:06:19.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:19.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 20 INFO: main: starting shard=1 2024-03-04T18:06:19.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.1 2024-03-04T18:06:19.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.1 2024-03-04T18:06:19.547 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:19.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 20 INFO: main: starting shard=2 2024-03-04T18:06:19.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.2 2024-03-04T18:06:19.548 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.547+0000 7f7cf65d8980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.2 2024-03-04T18:06:19.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.551+0000 7f7cf65d8980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T18:06:19.549 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.551+0000 7f7cf65d8980 20 INFO: main: done 2024-03-04T18:06:19.550 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.551+0000 7f7baafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:19.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.555+0000 7f7cf65d8980 20 remove_watcher() i=0 2024-03-04T18:06:19.555 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.555+0000 7f7cf65d8980 2 removed watcher, disabling cache 2024-03-04T18:06:19.557 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.559+0000 7f7cf65d8980 20 remove_watcher() i=1 2024-03-04T18:06:19.560 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.559+0000 7f7cf65d8980 20 remove_watcher() i=2 2024-03-04T18:06:19.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.563+0000 7f7cf65d8980 20 remove_watcher() i=3 2024-03-04T18:06:19.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.567+0000 7f7cf65d8980 20 remove_watcher() i=4 2024-03-04T18:06:19.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.567+0000 7f7cf65d8980 20 remove_watcher() i=5 2024-03-04T18:06:19.569 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.571+0000 7f7cf65d8980 20 remove_watcher() i=6 2024-03-04T18:06:19.572 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.571+0000 7f7cf65d8980 20 remove_watcher() i=7 2024-03-04T18:06:19.589 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:19.589 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:19.614 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:19.614 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:19.721 INFO:tasks.workunit.client.0.smithi137.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-03-04T18:06:19.721 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bi list --bucket myver --object z 2024-03-04T18:06:19.756 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:19.756 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:19.799 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.799+0000 7fb3709e2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.801 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.803+0000 7fb3709e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:19.801 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.803+0000 7fb3709e2980 20 realm 2024-03-04T18:06:19.801 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.803+0000 7fb3709e2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.803+0000 7fb3709e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:19.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.803+0000 7fb3709e2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:19.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.803+0000 7fb3709e2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.803+0000 7fb3709e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:19.802 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.803+0000 7fb3709e2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.803 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.803+0000 7fb3709e2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:19.803 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.803+0000 7fb3709e2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:19.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.807+0000 7fb3709e2980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:19.804 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:19.807+0000 7fb3709e2980 20 searching for the correct realm 2024-03-04T18:06:20.010 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.011+0000 7fb36ecfb640 0 -- 172.21.15.137:0/3554607956 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55a1f8c7ae50 msgr2=0x55a1f8c7d310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:20.020 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.023+0000 7fb36ecfb640 0 -- 172.21.15.137:0/3554607956 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x55a1f8c6e710 msgr2=0x55a1f8c70bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:20.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:20.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:20.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:20.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:20.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:20.032 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:20.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:20.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:20.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:20.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:20.033 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:20.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:20.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 zone default found 2024-03-04T18:06:20.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 4 Realm: () 2024-03-04T18:06:20.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:20.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:20.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:20.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 zonegroup default 2024-03-04T18:06:20.034 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:20.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:20.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:20.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:20.035 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.035+0000 7fb3709e2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:20.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.039+0000 7fb3709e2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:20.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.039+0000 7fb3709e2980 20 started sync module instance, tier type = 2024-03-04T18:06:20.036 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.039+0000 7fb3709e2980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:20.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.059+0000 7fb3709e2980 20 add_watcher() i=0 2024-03-04T18:06:20.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.059+0000 7fb3709e2980 20 add_watcher() i=1 2024-03-04T18:06:20.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.059+0000 7fb3709e2980 20 add_watcher() i=2 2024-03-04T18:06:20.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.059+0000 7fb3709e2980 20 add_watcher() i=3 2024-03-04T18:06:20.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.059+0000 7fb3709e2980 20 add_watcher() i=4 2024-03-04T18:06:20.058 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.059+0000 7fb3709e2980 20 add_watcher() i=5 2024-03-04T18:06:20.059 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.059+0000 7fb3709e2980 20 add_watcher() i=6 2024-03-04T18:06:20.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.063+0000 7fb3709e2980 20 add_watcher() i=7 2024-03-04T18:06:20.061 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.063+0000 7fb3709e2980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:20.065 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.067+0000 7fb224fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:20.066 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.067+0000 7fb3709e2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:20.066 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.067+0000 7fb3709e2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:20.066 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.067+0000 7fb3709e2980 5 note: GC not initialized 2024-03-04T18:06:20.066 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.067+0000 7fb19f7fe640 20 reqs_thread_entry: start 2024-03-04T18:06:20.066 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.067+0000 7fb3709e2980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:20.067 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.067+0000 7fb3709e2980 20 Filter name: none 2024-03-04T18:06:20.067 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.067+0000 7fb19d7fa640 20 reqs_thread_entry: start 2024-03-04T18:06:20.067 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.067+0000 7fb3709e2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:20.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:20.068 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 20 rados->read ofs=0 len=0 2024-03-04T18:06:20.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:20.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:20.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-03-04T18:06:20.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 20 INFO: main: starting shard=0 2024-03-04T18:06:20.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.0 2024-03-04T18:06:20.069 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.0 2024-03-04T18:06:20.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:20.070 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 20 INFO: main: starting shard=1 2024-03-04T18:06:20.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.1 2024-03-04T18:06:20.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.1 2024-03-04T18:06:20.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-03-04T18:06:20.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 20 INFO: main: starting shard=2 2024-03-04T18:06:20.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.2 2024-03-04T18:06:20.071 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.071+0000 7fb3709e2980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.2 2024-03-04T18:06:20.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.075+0000 7fb3709e2980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-03-04T18:06:20.072 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.075+0000 7fb3709e2980 20 INFO: main: done 2024-03-04T18:06:20.074 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.075+0000 7fb224fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:20.078 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.079+0000 7fb3709e2980 20 remove_watcher() i=0 2024-03-04T18:06:20.078 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.079+0000 7fb3709e2980 2 removed watcher, disabling cache 2024-03-04T18:06:20.080 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.083+0000 7fb3709e2980 20 remove_watcher() i=1 2024-03-04T18:06:20.082 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.083+0000 7fb3709e2980 20 remove_watcher() i=2 2024-03-04T18:06:20.084 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.087+0000 7fb3709e2980 20 remove_watcher() i=3 2024-03-04T18:06:20.086 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.087+0000 7fb3709e2980 20 remove_watcher() i=4 2024-03-04T18:06:20.088 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.091+0000 7fb3709e2980 20 remove_watcher() i=5 2024-03-04T18:06:20.091 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.091+0000 7fb3709e2980 20 remove_watcher() i=6 2024-03-04T18:06:20.094 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:20.095+0000 7fb3709e2980 20 remove_watcher() i=7 2024-03-04T18:06:20.112 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:20.112 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:20.149 INFO:tasks.workunit.client.0.smithi137.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-03-04T18:06:20.570 INFO:tasks.workunit.client.0.smithi137.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-03-04T18:06:20.978 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 17 --yes-i-really-mean-it 2024-03-04T18:06:21.006 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:21.006 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:21.046 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.047+0000 7fa9100ac980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.051+0000 7fa9100ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:21.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.051+0000 7fa9100ac980 20 realm 2024-03-04T18:06:21.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.051+0000 7fa9100ac980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.051+0000 7fa9100ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:21.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.051+0000 7fa9100ac980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:21.049 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.051+0000 7fa9100ac980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.051+0000 7fa9100ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:21.050 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.051+0000 7fa9100ac980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.051 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.051+0000 7fa9100ac980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:21.051 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.051+0000 7fa9100ac980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.051 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.051+0000 7fa9100ac980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:21.051 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.051+0000 7fa9100ac980 20 searching for the correct realm 2024-03-04T18:06:21.056 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.059+0000 7fa90d3c3640 0 -- 172.21.15.137:0/3197692884 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7fa9080dc740 msgr2=0x7fa9080dae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:21.272 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.271+0000 7fa90d3c3640 0 -- 172.21.15.137:0/3197692884 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x558f49812b90 msgr2=0x558f49814fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:21.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:21.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:21.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:21.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:21.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:21.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:21.286 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:21.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:21.287 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:21.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 zone default found 2024-03-04T18:06:21.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 4 Realm: () 2024-03-04T18:06:21.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:21.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:21.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:21.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 zonegroup default 2024-03-04T18:06:21.288 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.287+0000 7fa9100ac980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.291+0000 7fa9100ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:21.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.291+0000 7fa9100ac980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.291+0000 7fa9100ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:21.289 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.291+0000 7fa9100ac980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.291+0000 7fa9100ac980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:21.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.291+0000 7fa9100ac980 20 started sync module instance, tier type = 2024-03-04T18:06:21.290 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.291+0000 7fa9100ac980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:21.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.315+0000 7fa9100ac980 20 add_watcher() i=0 2024-03-04T18:06:21.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.315+0000 7fa9100ac980 20 add_watcher() i=1 2024-03-04T18:06:21.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.315+0000 7fa9100ac980 20 add_watcher() i=2 2024-03-04T18:06:21.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.315+0000 7fa9100ac980 20 add_watcher() i=3 2024-03-04T18:06:21.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.315+0000 7fa9100ac980 20 add_watcher() i=4 2024-03-04T18:06:21.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.315+0000 7fa9100ac980 20 add_watcher() i=5 2024-03-04T18:06:21.312 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.315+0000 7fa9100ac980 20 add_watcher() i=6 2024-03-04T18:06:21.314 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.315+0000 7fa9100ac980 20 add_watcher() i=7 2024-03-04T18:06:21.315 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.315+0000 7fa9100ac980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:21.319 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.319+0000 7fa7bf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:21.320 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.319+0000 7fa9100ac980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:21.320 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.319+0000 7fa9100ac980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:21.320 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 5 note: GC not initialized 2024-03-04T18:06:21.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa7be7fc640 20 reqs_thread_entry: start 2024-03-04T18:06:21.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:21.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 20 Filter name: none 2024-03-04T18:06:21.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa72ffff640 20 reqs_thread_entry: start 2024-03-04T18:06:21.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-03-04T18:06:21.321 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:21.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-03-04T18:06:21.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-03-04T18:06:21.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 : miss 2024-03-04T18:06:21.322 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:21.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:21.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 10 adding default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:21.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:21.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:21.323 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.323+0000 7fa9100ac980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 2024-03-04T18:06:21.337 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.339+0000 7fa9100ac980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:21.337 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.339+0000 7fa9100ac980 10 moving default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:21.337 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.339+0000 7fa9100ac980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:21.337 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.339+0000 7fa9100ac980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19, ofs0, ns] 2024-03-04T18:06:21.337 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.339+0000 7fa7f17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627857 cookie 94073901895056 notifier 7287 bl.length()=855 2024-03-04T18:06:21.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.339+0000 7fa7f17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:21.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.339+0000 7fa7f17fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:21.338 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.339+0000 7fa7f17fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:21.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): orig_sources= new_sources= 2024-03-04T18:06:21.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): potential sources added=[] removed=[] 2024-03-04T18:06:21.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): orig_dests= new_dests= 2024-03-04T18:06:21.340 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): potential dests added=[] removed=[] 2024-03-04T18:06:21.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.0 2024-03-04T18:06:21.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.0 2024-03-04T18:06:21.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.1 2024-03-04T18:06:21.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.1 2024-03-04T18:06:21.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.2 2024-03-04T18:06:21.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.2 2024-03-04T18:06:21.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.3 2024-03-04T18:06:21.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.3 2024-03-04T18:06:21.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.4 2024-03-04T18:06:21.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.4 2024-03-04T18:06:21.344 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.5 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.5 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.6 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.6 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.7 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.7 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.8 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.8 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.10 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.10 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.11 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.11 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.12 2024-03-04T18:06:21.345 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.12 2024-03-04T18:06:21.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.13 2024-03-04T18:06:21.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.13 2024-03-04T18:06:21.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.14 2024-03-04T18:06:21.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.14 2024-03-04T18:06:21.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.15 2024-03-04T18:06:21.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.15 2024-03-04T18:06:21.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.16 2024-03-04T18:06:21.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.16 2024-03-04T18:06:21.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.0 2024-03-04T18:06:21.346 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.343+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.0 2024-03-04T18:06:21.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.347+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.1 2024-03-04T18:06:21.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.347+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.1 2024-03-04T18:06:21.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.347+0000 7fa9100ac980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.2 2024-03-04T18:06:21.347 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.347+0000 7fa9100ac980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.2.2 2024-03-04T18:06:21.351 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.351+0000 7fa9100ac980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 : hit (requested=0x17, cached=0x17) 2024-03-04T18:06:21.351 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.351+0000 7fa9100ac980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 2024-03-04T18:06:21.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.355+0000 7fa9100ac980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:21.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.355+0000 7fa9100ac980 10 moving default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:21.354 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.355+0000 7fa9100ac980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:21.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.355+0000 7fa9100ac980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19, ofs0, ns] 2024-03-04T18:06:21.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.355+0000 7fa7f17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627858 cookie 94073901895056 notifier 7287 bl.length()=823 2024-03-04T18:06:21.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.355+0000 7fa7f17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:21.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.355+0000 7fa7f17fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:21.355 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.355+0000 7fa7f17fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:21.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.359+0000 7fa9100ac980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): orig_sources= new_sources= 2024-03-04T18:06:21.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.359+0000 7fa9100ac980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): potential sources added=[] removed=[] 2024-03-04T18:06:21.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.359+0000 7fa9100ac980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): orig_dests= new_dests= 2024-03-04T18:06:21.357 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.359+0000 7fa9100ac980 20 handle_bi_update(): bucket=:myver[f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19]): potential dests added=[] removed=[] 2024-03-04T18:06:21.360 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.363+0000 7fa9100ac980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-03-04T18:06:21.363 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.363+0000 7fa7bf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:21.367 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.367+0000 7fa9100ac980 20 remove_watcher() i=0 2024-03-04T18:06:21.367 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.367+0000 7fa9100ac980 2 removed watcher, disabling cache 2024-03-04T18:06:21.369 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.371+0000 7fa9100ac980 20 remove_watcher() i=1 2024-03-04T18:06:21.371 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.371+0000 7fa9100ac980 20 remove_watcher() i=2 2024-03-04T18:06:21.373 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.375+0000 7fa9100ac980 20 remove_watcher() i=3 2024-03-04T18:06:21.375 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.375+0000 7fa9100ac980 20 remove_watcher() i=4 2024-03-04T18:06:21.377 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.379+0000 7fa9100ac980 20 remove_watcher() i=5 2024-03-04T18:06:21.380 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.383+0000 7fa9100ac980 20 remove_watcher() i=6 2024-03-04T18:06:21.383 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.383+0000 7fa9100ac980 20 remove_watcher() i=7 2024-03-04T18:06:21.398 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket check unlinked --bucket myver --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-03-04T18:06:21.438 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:21.438 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:21.479 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.479+0000 7f3d7055d980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.483+0000 7f3d7055d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:21.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.483+0000 7f3d7055d980 20 realm 2024-03-04T18:06:21.482 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.483+0000 7f3d7055d980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.483+0000 7f3d7055d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:21.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.483+0000 7f3d7055d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:21.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.483+0000 7f3d7055d980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.483+0000 7f3d7055d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:21.483 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.483+0000 7f3d7055d980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.483+0000 7f3d7055d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:21.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.483+0000 7f3d7055d980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.484 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.487+0000 7f3d7055d980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:21.485 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.487+0000 7f3d7055d980 20 searching for the correct realm 2024-03-04T18:06:21.490 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.491+0000 7f3d6e876640 0 -- 172.21.15.137:0/3988002185 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f3d68061e40 msgr2=0x7f3d680710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:21.705 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.707+0000 7f3d6e876640 0 -- 172.21.15.137:0/3988002185 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55f0dd516700 msgr2=0x55f0dd518bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:21.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.715+0000 7f3d7055d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:21.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.715+0000 7f3d7055d980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:21.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.715+0000 7f3d7055d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:21.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.715+0000 7f3d7055d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:21.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.715+0000 7f3d7055d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:21.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.715+0000 7f3d7055d980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:21.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.715+0000 7f3d7055d980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.715+0000 7f3d7055d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:21.715 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.715+0000 7f3d7055d980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:21.716 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:21.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 20 zone default found 2024-03-04T18:06:21.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 4 Realm: () 2024-03-04T18:06:21.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:21.717 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:21.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:21.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 20 zonegroup default 2024-03-04T18:06:21.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:21.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:21.718 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:21.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 20 started sync module instance, tier type = 2024-03-04T18:06:21.719 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.719+0000 7f3d7055d980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:21.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.743+0000 7f3d7055d980 20 add_watcher() i=0 2024-03-04T18:06:21.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.743+0000 7f3d7055d980 20 add_watcher() i=1 2024-03-04T18:06:21.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.743+0000 7f3d7055d980 20 add_watcher() i=2 2024-03-04T18:06:21.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.743+0000 7f3d7055d980 20 add_watcher() i=3 2024-03-04T18:06:21.743 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.743+0000 7f3d7055d980 20 add_watcher() i=4 2024-03-04T18:06:21.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.743+0000 7f3d7055d980 20 add_watcher() i=5 2024-03-04T18:06:21.744 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.743+0000 7f3d7055d980 20 add_watcher() i=6 2024-03-04T18:06:21.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.747+0000 7f3d7055d980 20 add_watcher() i=7 2024-03-04T18:06:21.746 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.747+0000 7f3d7055d980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:21.750 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.751+0000 7f3c24fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:21.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.751+0000 7f3d7055d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:21.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.751+0000 7f3d7055d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:21.751 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.751+0000 7f3d7055d980 5 note: GC not initialized 2024-03-04T18:06:21.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.751+0000 7f3b9f7fe640 20 reqs_thread_entry: start 2024-03-04T18:06:21.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:21.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 20 Filter name: none 2024-03-04T18:06:21.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3b9d7fa640 20 reqs_thread_entry: start 2024-03-04T18:06:21.752 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-03-04T18:06:21.753 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:21.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-03-04T18:06:21.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-03-04T18:06:21.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 : miss 2024-03-04T18:06:21.754 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 20 rados->read ofs=0 len=0 2024-03-04T18:06:21.755 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:21.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 info.flags=0x17 2024-03-04T18:06:21.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 10 adding default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 to cache LRU end 2024-03-04T18:06:21.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:06:21.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-04T18:06:21.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19 2024-03-04T18:06:21.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.0 2024-03-04T18:06:21.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.755+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.0 2024-03-04T18:06:21.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.759+0000 7f3d7055d980 1 NOTICE: finished shard 0 (0 entries removed) 2024-03-04T18:06:21.756 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.759+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.1 2024-03-04T18:06:21.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.759+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.1 2024-03-04T18:06:21.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.759+0000 7f3d7055d980 1 NOTICE: finished shard 1 (0 entries removed) 2024-03-04T18:06:21.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.759+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.2 2024-03-04T18:06:21.757 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.759+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.2 2024-03-04T18:06:21.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.759+0000 7f3d7055d980 1 NOTICE: finished shard 2 (0 entries removed) 2024-03-04T18:06:21.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.759+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.3 2024-03-04T18:06:21.758 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.759+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.3 2024-03-04T18:06:21.760 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.759+0000 7f3d7055d980 1 NOTICE: finished shard 3 (0 entries removed) 2024-03-04T18:06:21.760 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.759+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.4 2024-03-04T18:06:21.760 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.759+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.4 2024-03-04T18:06:21.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.763+0000 7f3d7055d980 1 NOTICE: finished shard 4 (0 entries removed) 2024-03-04T18:06:21.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.763+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.5 2024-03-04T18:06:21.761 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.763+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.5 2024-03-04T18:06:21.762 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.763+0000 7f3d7055d980 1 NOTICE: finished shard 5 (0 entries removed) 2024-03-04T18:06:21.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.763+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.6 2024-03-04T18:06:21.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.763+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.6 2024-03-04T18:06:21.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.763+0000 7f3d7055d980 1 NOTICE: finished shard 6 (0 entries removed) 2024-03-04T18:06:21.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.763+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.7 2024-03-04T18:06:21.763 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.763+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.7 2024-03-04T18:06:21.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.767+0000 7f3d7055d980 1 NOTICE: finished shard 7 (0 entries removed) 2024-03-04T18:06:21.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.767+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.8 2024-03-04T18:06:21.764 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.767+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.8 2024-03-04T18:06:21.765 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.767+0000 7f3d7055d980 1 NOTICE: finished shard 8 (0 entries removed) 2024-03-04T18:06:21.765 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.767+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.765 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.767+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.767 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.767+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.10 2024-03-04T18:06:21.767 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.767+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.10 2024-03-04T18:06:21.767 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.767+0000 7f3d7055d980 1 NOTICE: finished shard 10 (0 entries removed) 2024-03-04T18:06:21.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.767+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.11 2024-03-04T18:06:21.768 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.767+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.11 2024-03-04T18:06:21.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.771+0000 7f3d7055d980 1 NOTICE: finished shard 11 (0 entries removed) 2024-03-04T18:06:21.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.771+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.12 2024-03-04T18:06:21.769 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.771+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.12 2024-03-04T18:06:21.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.771+0000 7f3d7055d980 1 NOTICE: finished shard 12 (0 entries removed) 2024-03-04T18:06:21.770 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.771+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.13 2024-03-04T18:06:21.771 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.771+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.13 2024-03-04T18:06:21.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.775+0000 7f3d7055d980 1 NOTICE: finished shard 13 (0 entries removed) 2024-03-04T18:06:21.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.775+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.14 2024-03-04T18:06:21.772 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.775+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.14 2024-03-04T18:06:21.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.775+0000 7f3d7055d980 1 NOTICE: finished shard 14 (0 entries removed) 2024-03-04T18:06:21.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.775+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.15 2024-03-04T18:06:21.773 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.775+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.15 2024-03-04T18:06:21.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.775+0000 7f3d7055d980 1 NOTICE: finished shard 15 (0 entries removed) 2024-03-04T18:06:21.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.775+0000 7f3d7055d980 10 bucket_obj is .dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.16 2024-03-04T18:06:21.774 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.775+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.16 2024-03-04T18:06:21.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.775+0000 7f3d7055d980 1 NOTICE: finished shard 16 (0 entries removed) 2024-03-04T18:06:21.775 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.775+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.776 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.779+0000 7f3d7055d980 20 get_obj_state: octx=0x55f0dd29b310 obj=myver:x state=0x55f0dd542328 s->prefetch_data=0 2024-03-04T18:06:21.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.779+0000 7f3d7055d980 20 get_obj_state: setting s->obj_tag to ulvgwpmis8fczmzmpd6q00fz9jl9q0j6 2024-03-04T18:06:21.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.779+0000 7f3d7055d980 20 get_obj_state_impl: setting s->olh_tag to dpqz08e2b4eqlvtx31vspx9h7fcfuqke 2024-03-04T18:06:21.777 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.779+0000 7f3d7055d980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=dpqz08e2b4eqlvtx31vspx9h7fcfuqke 2024-03-04T18:06:21.780 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.783+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.784 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.787+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.785 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.787+0000 7f3d7055d980 20 olh_log_entry: epoch=2 op=2 key=x[] 2024-03-04T18:06:21.785 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.787+0000 7f3d7055d980 20 olh_log_entry: epoch=2 op=3 key=x[rEk8tOcu27R7D.mwBTigZCUm.4HPsr6] 2024-03-04T18:06:21.785 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.787+0000 7f3d7055d980 20 get_obj_state: octx=0x55f0dd29b310 obj=myver:_:rEk8tOcu27R7D.mwBTigZCUm.4HPsr6_x state=0x55f0dd542e88 s->prefetch_data=0 2024-03-04T18:06:21.786 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.787+0000 7f3d7055d980 10 manifest: total_size = 11 2024-03-04T18:06:21.787 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.787+0000 7f3d7055d980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11773467500925669352 2024-03-04T18:06:21.787 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.787+0000 7f3d7055d980 20 get_obj_state: octx=0x55f0dd29b310 obj=myver:_:rEk8tOcu27R7D.mwBTigZCUm.4HPsr6_x state=0x55f0dd542e88 s->prefetch_data=0 2024-03-04T18:06:21.787 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.787+0000 7f3d7055d980 20 prepare_atomic_modification: state is not atomic. state=0x55f0dd542e88 2024-03-04T18:06:21.787 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.787+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.795+0000 7f3d7055d980 20 get_obj_state: octx=0x55f0dd29b310 obj=myver:x state=0x55f0dd542328 s->prefetch_data=0 2024-03-04T18:06:21.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.795+0000 7f3d7055d980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=dpqz08e2b4eqlvtx31vspx9h7fcfuqke 2024-03-04T18:06:21.796 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.799+0000 7f3c517fa640 20 handle_completion(): completion ok for obj=x[rEk8tOcu27R7D.mwBTigZCUm.4HPsr6] 2024-03-04T18:06:21.801 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.803+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.805 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.807+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.806 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.807+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.807+0000 7f3d7055d980 20 get_obj_state: octx=0x55f0dd29b310 obj=myver:y state=0x55f0dd541e28 s->prefetch_data=0 2024-03-04T18:06:21.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.807+0000 7f3d7055d980 20 get_obj_state: setting s->obj_tag to ami15lqq5dhnr3vz628a2v7pgk7e78k5 2024-03-04T18:06:21.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.807+0000 7f3d7055d980 20 get_obj_state_impl: setting s->olh_tag to 6md8ylq8f0l0xwjj2dqpsvimg7ptdojw 2024-03-04T18:06:21.807 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.807+0000 7f3d7055d980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=6md8ylq8f0l0xwjj2dqpsvimg7ptdojw 2024-03-04T18:06:21.809 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.811+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.813 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.815+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.814 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.815+0000 7f3d7055d980 20 olh_log_entry: epoch=2 op=2 key=y[] 2024-03-04T18:06:21.814 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.815+0000 7f3d7055d980 20 olh_log_entry: epoch=2 op=3 key=y[jxTBb9CxO4gp5ImQriEjF5G6t5ssmIF] 2024-03-04T18:06:21.814 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.815+0000 7f3d7055d980 20 get_obj_state: octx=0x55f0dd29b310 obj=myver:_:jxTBb9CxO4gp5ImQriEjF5G6t5ssmIF_y state=0x55f0dd5774c8 s->prefetch_data=0 2024-03-04T18:06:21.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.815+0000 7f3d7055d980 10 manifest: total_size = 22 2024-03-04T18:06:21.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.815+0000 7f3d7055d980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.1505572319237024396 2024-03-04T18:06:21.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.815+0000 7f3d7055d980 20 get_obj_state: octx=0x55f0dd29b310 obj=myver:_:jxTBb9CxO4gp5ImQriEjF5G6t5ssmIF_y state=0x55f0dd5774c8 s->prefetch_data=0 2024-03-04T18:06:21.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.815+0000 7f3d7055d980 20 prepare_atomic_modification: state is not atomic. state=0x55f0dd5774c8 2024-03-04T18:06:21.815 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.815+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.823+0000 7f3d7055d980 20 get_obj_state: octx=0x55f0dd29b310 obj=myver:y state=0x55f0dd541e28 s->prefetch_data=0 2024-03-04T18:06:21.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.823+0000 7f3d7055d980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=6md8ylq8f0l0xwjj2dqpsvimg7ptdojw 2024-03-04T18:06:21.824 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.827+0000 7f3c51ffb640 20 handle_completion(): completion ok for obj=y[jxTBb9CxO4gp5ImQriEjF5G6t5ssmIF] 2024-03-04T18:06:21.827 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.827+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.830 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.831+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.831 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.831+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.832 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.835+0000 7f3d7055d980 20 get_obj_state: octx=0x55f0dd29b310 obj=myver:z state=0x55f0dd548c98 s->prefetch_data=0 2024-03-04T18:06:21.833 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.835+0000 7f3d7055d980 20 get_obj_state: setting s->obj_tag to l9npuh4vcnhfb6hq2pw8bgop6i7exfoe 2024-03-04T18:06:21.833 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.835+0000 7f3d7055d980 20 get_obj_state_impl: setting s->olh_tag to 060l02vijbrg28d5wcnif50xi6o3jqi7 2024-03-04T18:06:21.833 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.835+0000 7f3d7055d980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=060l02vijbrg28d5wcnif50xi6o3jqi7 2024-03-04T18:06:21.836 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.835+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.839 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.839+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.843+0000 7f3d7055d980 20 olh_log_entry: epoch=2 op=2 key=z[] 2024-03-04T18:06:21.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.843+0000 7f3d7055d980 20 olh_log_entry: epoch=2 op=3 key=z[PYpyVEtFytLjNGnPtT8FvkzZ64UJ7TY] 2024-03-04T18:06:21.840 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.843+0000 7f3d7055d980 20 get_obj_state: octx=0x55f0dd29b310 obj=myver:_:PYpyVEtFytLjNGnPtT8FvkzZ64UJ7TY_z state=0x55f0dd5774c8 s->prefetch_data=0 2024-03-04T18:06:21.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.843+0000 7f3d7055d980 10 manifest: total_size = 12 2024-03-04T18:06:21.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.843+0000 7f3d7055d980 20 get_obj_state: setting s->obj_tag to f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9739412918939188935 2024-03-04T18:06:21.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.843+0000 7f3d7055d980 20 get_obj_state: octx=0x55f0dd29b310 obj=myver:_:PYpyVEtFytLjNGnPtT8FvkzZ64UJ7TY_z state=0x55f0dd5774c8 s->prefetch_data=0 2024-03-04T18:06:21.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.843+0000 7f3d7055d980 20 prepare_atomic_modification: state is not atomic. state=0x55f0dd5774c8 2024-03-04T18:06:21.841 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.843+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.850 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.851+0000 7f3d7055d980 20 get_obj_state: octx=0x55f0dd29b310 obj=myver:z state=0x55f0dd548c98 s->prefetch_data=0 2024-03-04T18:06:21.850 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.851+0000 7f3d7055d980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=060l02vijbrg28d5wcnif50xi6o3jqi7 2024-03-04T18:06:21.850 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.851+0000 7f3c517fa640 20 handle_completion(): completion ok for obj=z[PYpyVEtFytLjNGnPtT8FvkzZ64UJ7TY] 2024-03-04T18:06:21.854 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.855+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.857 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.859+0000 7f3d7055d980 20 bucket index object: :.dir.f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.19.3.9 2024-03-04T18:06:21.857 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.859+0000 7f3d7055d980 1 NOTICE: finished shard 9 (3 entries removed) 2024-03-04T18:06:21.857 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.859+0000 7f3d7055d980 1 NOTICE: finished all shards (3 entries removed) 2024-03-04T18:06:21.859 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.859+0000 7f3c24fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:21.864 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.863+0000 7f3d7055d980 20 remove_watcher() i=0 2024-03-04T18:06:21.864 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.863+0000 7f3d7055d980 2 removed watcher, disabling cache 2024-03-04T18:06:21.866 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.867+0000 7f3d7055d980 20 remove_watcher() i=1 2024-03-04T18:06:21.868 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.871+0000 7f3d7055d980 20 remove_watcher() i=2 2024-03-04T18:06:21.870 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.871+0000 7f3d7055d980 20 remove_watcher() i=3 2024-03-04T18:06:21.873 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.875+0000 7f3d7055d980 20 remove_watcher() i=4 2024-03-04T18:06:21.875 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.875+0000 7f3d7055d980 20 remove_watcher() i=5 2024-03-04T18:06:21.878 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.879+0000 7f3d7055d980 20 remove_watcher() i=6 2024-03-04T18:06:21.881 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:21.883+0000 7f3d7055d980 20 remove_watcher() i=7 2024-03-04T18:06:21.905 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:21.905 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:22.006 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:22.006 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:22.249 INFO:tasks.workunit.client.0.smithi137.stderr:radosgw-admin bucket stats --bucket myver 2024-03-04T18:06:22.282 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:22.282 INFO:tasks.workunit.client.0.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:22.326 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.327+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.331+0000 7fb7a85c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:22.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.331+0000 7fb7a85c4980 20 realm 2024-03-04T18:06:22.328 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.331+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.331+0000 7fb7a85c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:22.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.331+0000 7fb7a85c4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:22.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.331+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.331+0000 7fb7a85c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:22.329 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.331+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.331+0000 7fb7a85c4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:22.330 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.331+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.331+0000 7fb7a85c4980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:22.331 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.331+0000 7fb7a85c4980 20 searching for the correct realm 2024-03-04T18:06:22.336 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.339+0000 7fb7a68dd640 0 -- 172.21.15.137:0/760978171 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fb7a0018040 msgr2=0x7fb7a005b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:22.553 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.555+0000 7fb7a68dd640 0 -- 172.21.15.137:0/760978171 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x558f6eed3fa0 msgr2=0x558f6eed6440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:22.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.563+0000 7fb7a85c4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:22.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.563+0000 7fb7a85c4980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:22.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.563+0000 7fb7a85c4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:22.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.563+0000 7fb7a85c4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:22.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.563+0000 7fb7a85c4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:22.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.563+0000 7fb7a85c4980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:22.562 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.563+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.563+0000 7fb7a85c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:22.563 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.563+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.563+0000 7fb7a85c4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:22.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.563+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:22.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 20 zone default found 2024-03-04T18:06:22.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 4 Realm: () 2024-03-04T18:06:22.564 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:22.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:22.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:22.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 20 zonegroup default 2024-03-04T18:06:22.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:22.565 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:22.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.566 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:22.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 20 started sync module instance, tier type = 2024-03-04T18:06:22.567 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.567+0000 7fb7a85c4980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:22.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.591+0000 7fb7a85c4980 20 add_watcher() i=0 2024-03-04T18:06:22.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.591+0000 7fb7a85c4980 20 add_watcher() i=1 2024-03-04T18:06:22.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.591+0000 7fb7a85c4980 20 add_watcher() i=2 2024-03-04T18:06:22.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.591+0000 7fb7a85c4980 20 add_watcher() i=3 2024-03-04T18:06:22.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.591+0000 7fb7a85c4980 20 add_watcher() i=4 2024-03-04T18:06:22.589 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.591+0000 7fb7a85c4980 20 add_watcher() i=5 2024-03-04T18:06:22.590 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.591+0000 7fb7a85c4980 20 add_watcher() i=6 2024-03-04T18:06:22.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.591+0000 7fb7a85c4980 20 add_watcher() i=7 2024-03-04T18:06:22.592 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.591+0000 7fb7a85c4980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:22.596 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.595+0000 7fb65cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:22.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.599+0000 7fb7a85c4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:22.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.599+0000 7fb7a85c4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:22.597 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.599+0000 7fb7a85c4980 5 note: GC not initialized 2024-03-04T18:06:22.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.599+0000 7fb5d77fe640 20 reqs_thread_entry: start 2024-03-04T18:06:22.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.599+0000 7fb7a85c4980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:22.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.599+0000 7fb7a85c4980 20 Filter name: none 2024-03-04T18:06:22.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.599+0000 7fb5d57fa640 20 reqs_thread_entry: start 2024-03-04T18:06:22.598 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.599+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.599+0000 7fb7a85c4980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:22.599 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.599+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.603+0000 7fb7a85c4980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:22.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.603+0000 7fb7a85c4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:22.600 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.603+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.601 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.603+0000 7fb7a85c4980 20 rados_obj.operate() r=0 bl.length=197 2024-03-04T18:06:22.601 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.603+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.602 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.603+0000 7fb7a85c4980 20 rados_obj.operate() r=0 bl.length=369 2024-03-04T18:06:22.603 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.603+0000 7fb7a85c4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-04T18:06:22.605 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.607+0000 7fb7a85c4980 20 rados->read ofs=0 len=0 2024-03-04T18:06:22.606 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.607+0000 7fb7a85c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:22.607 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.607+0000 7fb65cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:22.611 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.611+0000 7fb7a85c4980 20 remove_watcher() i=0 2024-03-04T18:06:22.612 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.611+0000 7fb7a85c4980 2 removed watcher, disabling cache 2024-03-04T18:06:22.614 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.615+0000 7fb7a85c4980 20 remove_watcher() i=1 2024-03-04T18:06:22.616 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.619+0000 7fb7a85c4980 20 remove_watcher() i=2 2024-03-04T18:06:22.619 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.619+0000 7fb7a85c4980 20 remove_watcher() i=3 2024-03-04T18:06:22.621 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.623+0000 7fb7a85c4980 20 remove_watcher() i=4 2024-03-04T18:06:22.623 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.623+0000 7fb7a85c4980 20 remove_watcher() i=5 2024-03-04T18:06:22.626 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.627+0000 7fb7a85c4980 20 remove_watcher() i=6 2024-03-04T18:06:22.628 INFO:tasks.workunit.client.0.smithi137.stderr:2024-03-04T18:06:22.631+0000 7fb7a85c4980 20 remove_watcher() i=7 2024-03-04T18:06:22.643 INFO:tasks.workunit.client.0.smithi137.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2024-03-04T18:06:22.643 INFO:tasks.workunit.client.0.smithi137.stderr:Deleting bucket a-bucket 2024-03-04T18:06:22.644 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:22.645 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:22.675 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:22.675 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:22.780 INFO:tasks.workunit.client.0.smithi137.stderr:Deleting bucket myver 2024-03-04T18:06:22.782 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:22.782 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:22.912 INFO:tasks.workunit.client.0.smithi137.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings 2024-03-04T18:06:22.912 INFO:tasks.workunit.client.0.smithi137.stderr: warnings.warn( 2024-03-04T18:06:23.361 INFO:tasks.workunit.client.0.smithi137.stderr:Completed resharding tests 2024-03-04T18:06:23.362 INFO:tasks.workunit.client.0.smithi137.stdout:Could not connect to the endpoint URL: "http://localhost:80/" 2024-03-04T18:06:23.362 INFO:tasks.workunit.client.0.smithi137.stdout:Could not connect to the endpoint URL: "http://localhost:8000/" 2024-03-04T18:06:23.362 INFO:tasks.workunit.client.0.smithi137.stdout:connected to https://localhost:443 2024-03-04T18:06:23.472 INFO:tasks.workunit.client.0.smithi137.stderr:+ deactivate 2024-03-04T18:06:23.472 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-03-04T18:06:23.472 INFO:tasks.workunit.client.0.smithi137.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-03-04T18:06:23.472 INFO:tasks.workunit.client.0.smithi137.stderr:+ export PATH 2024-03-04T18:06:23.473 INFO:tasks.workunit.client.0.smithi137.stderr:+ unset _OLD_VIRTUAL_PATH 2024-03-04T18:06:23.473 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' -n '' ']' 2024-03-04T18:06:23.473 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-03-04T18:06:23.473 INFO:tasks.workunit.client.0.smithi137.stderr:+ hash -r 2024-03-04T18:06:23.473 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' -n '' ']' 2024-03-04T18:06:23.473 INFO:tasks.workunit.client.0.smithi137.stderr:+ unset VIRTUAL_ENV 2024-03-04T18:06:23.473 INFO:tasks.workunit.client.0.smithi137.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-03-04T18:06:23.473 INFO:tasks.workunit.client.0.smithi137.stderr:+ '[' '!' '' = nondestructive ']' 2024-03-04T18:06:23.473 INFO:tasks.workunit.client.0.smithi137.stderr:+ unset -f deactivate 2024-03-04T18:06:23.473 INFO:tasks.workunit.client.0.smithi137.stderr:+ echo OK. 2024-03-04T18:06:23.474 INFO:tasks.workunit.client.0.smithi137.stdout:OK. 2024-03-04T18:06:23.475 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T18:06:23.475 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T18:06:23.528 INFO:tasks.workunit:Stopping ['rgw/run-reshard.sh'] on client.0... 2024-03-04T18:06:23.528 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-04T18:06:23.957 DEBUG:teuthology.parallel:result is None 2024-03-04T18:06:23.957 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T18:06:23.969 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T18:06:23.969 DEBUG:teuthology.orchestra.run.smithi137:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T18:06:24.016 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T18:06:24.017 INFO:teuthology.run_tasks:Running task s3tests-java... 2024-03-04T18:06:24.034 DEBUG:tasks.s3tests_java:S3 Tests Java: __INIT__ 2024-03-04T18:06:24.034 DEBUG:tasks.s3tests_java:S3 Tests Java: SETUP 2024-03-04T18:06:24.034 INFO:tasks.s3tests_java:S3 Tests Java: Downloading test suite... 2024-03-04T18:06:24.035 DEBUG:teuthology.orchestra.run.smithi137:> git clone -b ceph-master https://github.com/ceph/java_s3tests.git /home/ubuntu/cephtest/s3-tests-java 2024-03-04T18:06:24.064 INFO:teuthology.orchestra.run.smithi137.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-java'... 2024-03-04T18:06:29.593 INFO:tasks.s3tests_java:S3 Tests Java: Installing required packages... 2024-03-04T18:06:29.594 DEBUG:teuthology.orchestra.run.smithi137:> /home/ubuntu/cephtest/s3-tests-java/bootstrap.sh 2024-03-04T18:06:29.685 INFO:teuthology.orchestra.run.smithi137.stderr:/home/ubuntu/cephtest/s3-tests-java/bootstrap.sh: missing required DEB packages. Installing via sudo. 2024-03-04T18:06:29.742 INFO:teuthology.orchestra.run.smithi137.stdout:Reading package lists... 2024-03-04T18:06:29.939 INFO:teuthology.orchestra.run.smithi137.stdout:Building dependency tree... 2024-03-04T18:06:29.940 INFO:teuthology.orchestra.run.smithi137.stdout:Reading state information... 2024-03-04T18:06:30.114 INFO:teuthology.orchestra.run.smithi137.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T18:06:30.114 INFO:teuthology.orchestra.run.smithi137.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T18:06:30.114 INFO:teuthology.orchestra.run.smithi137.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T18:06:30.114 INFO:teuthology.orchestra.run.smithi137.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T18:06:30.114 INFO:teuthology.orchestra.run.smithi137.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T18:06:30.115 INFO:teuthology.orchestra.run.smithi137.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T18:06:30.115 INFO:teuthology.orchestra.run.smithi137.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T18:06:30.115 INFO:teuthology.orchestra.run.smithi137.stdout:The following additional packages will be installed: 2024-03-04T18:06:30.116 INFO:teuthology.orchestra.run.smithi137.stdout: libgail-common libgail18 libgtk2.0-0 libgtk2.0-bin libgtk2.0-common 2024-03-04T18:06:30.116 INFO:teuthology.orchestra.run.smithi137.stdout: openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless 2024-03-04T18:06:30.117 INFO:teuthology.orchestra.run.smithi137.stdout:Suggested packages: 2024-03-04T18:06:30.118 INFO:teuthology.orchestra.run.smithi137.stdout: gvfs openjdk-8-demo openjdk-8-source visualvm libnss-mdns fonts-nanum 2024-03-04T18:06:30.118 INFO:teuthology.orchestra.run.smithi137.stdout: fonts-ipafont-gothic fonts-ipafont-mincho fonts-wqy-microhei 2024-03-04T18:06:30.118 INFO:teuthology.orchestra.run.smithi137.stdout: fonts-wqy-zenhei fonts-indic 2024-03-04T18:06:30.146 INFO:teuthology.orchestra.run.smithi137.stdout:The following NEW packages will be installed: 2024-03-04T18:06:30.147 INFO:teuthology.orchestra.run.smithi137.stdout: libgail-common libgail18 libgtk2.0-0 libgtk2.0-bin libgtk2.0-common 2024-03-04T18:06:30.147 INFO:teuthology.orchestra.run.smithi137.stdout: openjdk-8-jdk openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless 2024-03-04T18:06:30.411 INFO:teuthology.orchestra.run.smithi137.stdout:0 upgraded, 9 newly installed, 0 to remove and 275 not upgraded. 2024-03-04T18:06:30.411 INFO:teuthology.orchestra.run.smithi137.stdout:Need to get 46.1 MB of archives. 2024-03-04T18:06:30.411 INFO:teuthology.orchestra.run.smithi137.stdout:After this operation, 155 MB of additional disk space will be used. 2024-03-04T18:06:30.411 INFO:teuthology.orchestra.run.smithi137.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-common all 2.24.33-2ubuntu2 [125 kB] 2024-03-04T18:06:30.774 INFO:teuthology.orchestra.run.smithi137.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-0 amd64 2.24.33-2ubuntu2 [2,037 kB] 2024-03-04T18:06:31.132 INFO:teuthology.orchestra.run.smithi137.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgail18 amd64 2.24.33-2ubuntu2 [15.9 kB] 2024-03-04T18:06:31.133 INFO:teuthology.orchestra.run.smithi137.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgail-common amd64 2.24.33-2ubuntu2 [132 kB] 2024-03-04T18:06:31.140 INFO:teuthology.orchestra.run.smithi137.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-bin amd64 2.24.33-2ubuntu2 [7,932 B] 2024-03-04T18:06:31.140 INFO:teuthology.orchestra.run.smithi137.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jre-headless amd64 8u392-ga-1~22.04 [30.8 MB] 2024-03-04T18:06:32.331 INFO:teuthology.orchestra.run.smithi137.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jre amd64 8u392-ga-1~22.04 [75.5 kB] 2024-03-04T18:06:32.334 INFO:teuthology.orchestra.run.smithi137.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jdk-headless amd64 8u392-ga-1~22.04 [8,863 kB] 2024-03-04T18:06:32.602 INFO:teuthology.orchestra.run.smithi137.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jdk amd64 8u392-ga-1~22.04 [4,006 kB] 2024-03-04T18:06:33.022 INFO:teuthology.orchestra.run.smithi137.stderr:debconf: unable to initialize frontend: Dialog 2024-03-04T18:06:33.022 INFO:teuthology.orchestra.run.smithi137.stderr:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-03-04T18:06:33.022 INFO:teuthology.orchestra.run.smithi137.stderr:debconf: falling back to frontend: Readline 2024-03-04T18:06:33.032 INFO:teuthology.orchestra.run.smithi137.stderr:debconf: unable to initialize frontend: Readline 2024-03-04T18:06:33.032 INFO:teuthology.orchestra.run.smithi137.stderr:debconf: (This frontend requires a controlling tty.) 2024-03-04T18:06:33.032 INFO:teuthology.orchestra.run.smithi137.stderr:debconf: falling back to frontend: Teletype 2024-03-04T18:06:33.036 INFO:teuthology.orchestra.run.smithi137.stderr:dpkg-preconfigure: unable to re-open stdin: 2024-03-04T18:06:33.068 INFO:teuthology.orchestra.run.smithi137.stdout:Fetched 46.1 MB in 3s (18.0 MB/s) 2024-03-04T18:06:33.176 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libgtk2.0-common. 2024-03-04T18:06:33.258 INFO:teuthology.orchestra.run.smithi137.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174558 files and directories currently installed.) 2024-03-04T18:06:33.264 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../0-libgtk2.0-common_2.24.33-2ubuntu2_all.deb ... 2024-03-04T18:06:33.307 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libgtk2.0-common (2.24.33-2ubuntu2) ... 2024-03-04T18:06:33.720 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libgtk2.0-0:amd64. 2024-03-04T18:06:33.747 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../1-libgtk2.0-0_2.24.33-2ubuntu2_amd64.deb ... 2024-03-04T18:06:33.828 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libgtk2.0-0:amd64 (2.24.33-2ubuntu2) ... 2024-03-04T18:06:34.274 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libgail18:amd64. 2024-03-04T18:06:34.303 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../2-libgail18_2.24.33-2ubuntu2_amd64.deb ... 2024-03-04T18:06:34.340 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libgail18:amd64 (2.24.33-2ubuntu2) ... 2024-03-04T18:06:34.593 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libgail-common:amd64. 2024-03-04T18:06:34.609 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../3-libgail-common_2.24.33-2ubuntu2_amd64.deb ... 2024-03-04T18:06:34.651 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libgail-common:amd64 (2.24.33-2ubuntu2) ... 2024-03-04T18:06:34.920 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libgtk2.0-bin. 2024-03-04T18:06:34.936 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../4-libgtk2.0-bin_2.24.33-2ubuntu2_amd64.deb ... 2024-03-04T18:06:34.978 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libgtk2.0-bin (2.24.33-2ubuntu2) ... 2024-03-04T18:06:35.357 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package openjdk-8-jre-headless:amd64. 2024-03-04T18:06:35.385 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../5-openjdk-8-jre-headless_8u392-ga-1~22.04_amd64.deb ... 2024-03-04T18:06:35.435 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking openjdk-8-jre-headless:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:06:37.054 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package openjdk-8-jre:amd64. 2024-03-04T18:06:37.082 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../6-openjdk-8-jre_8u392-ga-1~22.04_amd64.deb ... 2024-03-04T18:06:37.124 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking openjdk-8-jre:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:06:37.584 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package openjdk-8-jdk-headless:amd64. 2024-03-04T18:06:37.611 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../7-openjdk-8-jdk-headless_8u392-ga-1~22.04_amd64.deb ... 2024-03-04T18:06:37.653 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking openjdk-8-jdk-headless:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:06:38.403 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package openjdk-8-jdk:amd64. 2024-03-04T18:06:38.420 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../8-openjdk-8-jdk_8u392-ga-1~22.04_amd64.deb ... 2024-03-04T18:06:38.456 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking openjdk-8-jdk:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:06:38.798 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up openjdk-8-jre-headless:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:06:39.844 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/orbd to provide /usr/bin/orbd (orbd) in auto mode 2024-03-04T18:06:39.870 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/servertool to provide /usr/bin/servertool (servertool) in auto mode 2024-03-04T18:06:39.897 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/tnameserv to provide /usr/bin/tnameserv (tnameserv) in auto mode 2024-03-04T18:06:40.312 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libgtk2.0-common (2.24.33-2ubuntu2) ... 2024-03-04T18:06:40.455 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libgtk2.0-0:amd64 (2.24.33-2ubuntu2) ... 2024-03-04T18:06:40.848 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up openjdk-8-jre:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:06:40.941 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/policytool to provide /usr/bin/policytool (policytool) in auto mode 2024-03-04T18:06:41.016 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up openjdk-8-jdk-headless:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:06:41.109 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/clhsdb to provide /usr/bin/clhsdb (clhsdb) in auto mode 2024-03-04T18:06:41.129 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/extcheck to provide /usr/bin/extcheck (extcheck) in auto mode 2024-03-04T18:06:41.154 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/hsdb to provide /usr/bin/hsdb (hsdb) in auto mode 2024-03-04T18:06:41.181 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/idlj to provide /usr/bin/idlj (idlj) in auto mode 2024-03-04T18:06:41.306 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/javah to provide /usr/bin/javah (javah) in auto mode 2024-03-04T18:06:41.469 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jhat to provide /usr/bin/jhat (jhat) in auto mode 2024-03-04T18:06:41.594 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jsadebugd to provide /usr/bin/jsadebugd (jsadebugd) in auto mode 2024-03-04T18:06:41.679 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/native2ascii to provide /usr/bin/native2ascii (native2ascii) in auto mode 2024-03-04T18:06:41.730 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/schemagen to provide /usr/bin/schemagen (schemagen) in auto mode 2024-03-04T18:06:41.773 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsgen to provide /usr/bin/wsgen (wsgen) in auto mode 2024-03-04T18:06:41.797 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsimport to provide /usr/bin/wsimport (wsimport) in auto mode 2024-03-04T18:06:41.823 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/xjc to provide /usr/bin/xjc (xjc) in auto mode 2024-03-04T18:06:41.887 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libgail18:amd64 (2.24.33-2ubuntu2) ... 2024-03-04T18:06:42.014 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libgtk2.0-bin (2.24.33-2ubuntu2) ... 2024-03-04T18:06:42.131 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up openjdk-8-jdk:amd64 (8u392-ga-1~22.04) ... 2024-03-04T18:06:42.207 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/appletviewer to provide /usr/bin/appletviewer (appletviewer) in auto mode 2024-03-04T18:06:42.300 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libgail-common:amd64 (2.24.33-2ubuntu2) ... 2024-03-04T18:06:42.427 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T18:06:43.453 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-04T18:06:43.561 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-04T18:06:43.871 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-04T18:06:44.325 INFO:teuthology.orchestra.run.smithi137.stdout: Gradle will be installed under /opt/gradle 2024-03-04T18:06:44.359 INFO:teuthology.orchestra.run.smithi137.stderr:--2024-03-04 18:06:44-- https://services.gradle.org/distributions/gradle-6.0.1-bin.zip 2024-03-04T18:06:44.363 INFO:teuthology.orchestra.run.smithi137.stderr:Resolving services.gradle.org (services.gradle.org)... 104.17.128.37, 104.17.129.37, 2606:4700::6811:8025, ... 2024-03-04T18:06:44.373 INFO:teuthology.orchestra.run.smithi137.stderr:Connecting to services.gradle.org (services.gradle.org)|104.17.128.37|:443... connected. 2024-03-04T18:06:44.421 INFO:teuthology.orchestra.run.smithi137.stderr:HTTP request sent, awaiting response... 301 Moved Permanently 2024-03-04T18:06:44.421 INFO:teuthology.orchestra.run.smithi137.stderr:Location: https://github.com/gradle/gradle-distributions/releases/download/v6.0.1/gradle-6.0.1-bin.zip [following] 2024-03-04T18:06:44.421 INFO:teuthology.orchestra.run.smithi137.stderr:--2024-03-04 18:06:44-- https://github.com/gradle/gradle-distributions/releases/download/v6.0.1/gradle-6.0.1-bin.zip 2024-03-04T18:06:44.422 INFO:teuthology.orchestra.run.smithi137.stderr:Resolving github.com (github.com)... 140.82.112.4 2024-03-04T18:06:44.433 INFO:teuthology.orchestra.run.smithi137.stderr:Connecting to github.com (github.com)|140.82.112.4|:443... connected. 2024-03-04T18:06:44.477 INFO:teuthology.orchestra.run.smithi137.stderr:HTTP request sent, awaiting response... 302 Found 2024-03-04T18:06:44.477 INFO:teuthology.orchestra.run.smithi137.stderr:Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/696192900/8f418ba1-c336-448c-95c6-5180382113cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAVCODYLSA53PQK4ZA%2F20240304%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240304T180625Z&X-Amz-Expires=300&X-Amz-Signature=18b24a4e3be041394ab5f04c4b925ab77003dce6a5ad8b478a9f10a31585849b&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=696192900&response-content-disposition=attachment%3B%20filename%3Dgradle-6.0.1-bin.zip&response-content-type=application%2Foctet-stream [following] 2024-03-04T18:06:44.477 INFO:teuthology.orchestra.run.smithi137.stderr:--2024-03-04 18:06:44-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/696192900/8f418ba1-c336-448c-95c6-5180382113cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAVCODYLSA53PQK4ZA%2F20240304%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240304T180625Z&X-Amz-Expires=300&X-Amz-Signature=18b24a4e3be041394ab5f04c4b925ab77003dce6a5ad8b478a9f10a31585849b&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=696192900&response-content-disposition=attachment%3B%20filename%3Dgradle-6.0.1-bin.zip&response-content-type=application%2Foctet-stream 2024-03-04T18:06:44.479 INFO:teuthology.orchestra.run.smithi137.stderr:Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.109.133, 185.199.108.133, 185.199.111.133, ... 2024-03-04T18:06:44.494 INFO:teuthology.orchestra.run.smithi137.stderr:Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.109.133|:443... connected. 2024-03-04T18:06:44.581 INFO:teuthology.orchestra.run.smithi137.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-04T18:06:44.581 INFO:teuthology.orchestra.run.smithi137.stderr:Length: 95129672 (91M) [application/octet-stream] 2024-03-04T18:06:44.581 INFO:teuthology.orchestra.run.smithi137.stderr:Saving to: 'gradle-6.0.1-bin.zip' 2024-03-04T18:06:44.581 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-03-04T18:06:44.596 INFO:teuthology.orchestra.run.smithi137.stderr: 0K .......... .......... .......... .......... .......... 0% 3.04M 30s 2024-03-04T18:06:44.609 INFO:teuthology.orchestra.run.smithi137.stderr: 50K .......... .......... .......... .......... .......... 0% 3.85M 27s 2024-03-04T18:06:44.613 INFO:teuthology.orchestra.run.smithi137.stderr: 100K .......... .......... .......... .......... .......... 0% 14.4M 20s 2024-03-04T18:06:44.615 INFO:teuthology.orchestra.run.smithi137.stderr: 150K .......... .......... .......... .......... .......... 0% 21.9M 16s 2024-03-04T18:06:44.623 INFO:teuthology.orchestra.run.smithi137.stderr: 200K .......... .......... .......... .......... .......... 0% 5.64M 16s 2024-03-04T18:06:44.625 INFO:teuthology.orchestra.run.smithi137.stderr: 250K .......... .......... .......... .......... .......... 0% 30.7M 14s 2024-03-04T18:06:44.627 INFO:teuthology.orchestra.run.smithi137.stderr: 300K .......... .......... .......... .......... .......... 0% 26.0M 12s 2024-03-04T18:06:44.629 INFO:teuthology.orchestra.run.smithi137.stderr: 350K .......... .......... .......... .......... .......... 0% 23.4M 11s 2024-03-04T18:06:44.631 INFO:teuthology.orchestra.run.smithi137.stderr: 400K .......... .......... .......... .......... .......... 0% 41.9M 10s 2024-03-04T18:06:44.632 INFO:teuthology.orchestra.run.smithi137.stderr: 450K .......... .......... .......... .......... .......... 0% 30.1M 9s 2024-03-04T18:06:44.639 INFO:teuthology.orchestra.run.smithi137.stderr: 500K .......... .......... .......... .......... .......... 0% 7.79M 10s 2024-03-04T18:06:44.640 INFO:teuthology.orchestra.run.smithi137.stderr: 550K .......... .......... .......... .......... .......... 0% 40.9M 9s 2024-03-04T18:06:44.640 INFO:teuthology.orchestra.run.smithi137.stderr: 600K .......... .......... .......... .......... .......... 0% 103M 8s 2024-03-04T18:06:44.647 INFO:teuthology.orchestra.run.smithi137.stderr: 650K .......... .......... .......... .......... .......... 0% 28.5M 8s 2024-03-04T18:06:44.647 INFO:teuthology.orchestra.run.smithi137.stderr: 700K .......... .......... .......... .......... .......... 0% 50.2M 8s 2024-03-04T18:06:44.647 INFO:teuthology.orchestra.run.smithi137.stderr: 750K .......... .......... .......... .......... .......... 0% 57.9M 7s 2024-03-04T18:06:44.647 INFO:teuthology.orchestra.run.smithi137.stderr: 800K .......... .......... .......... .......... .......... 0% 45.2M 7s 2024-03-04T18:06:44.648 INFO:teuthology.orchestra.run.smithi137.stderr: 850K .......... .......... .......... .......... .......... 0% 94.5M 7s 2024-03-04T18:06:44.648 INFO:teuthology.orchestra.run.smithi137.stderr: 900K .......... .......... .......... .......... .......... 1% 121M 6s 2024-03-04T18:06:44.648 INFO:teuthology.orchestra.run.smithi137.stderr: 950K .......... .......... .......... .......... .......... 1% 76.1M 6s 2024-03-04T18:06:44.658 INFO:teuthology.orchestra.run.smithi137.stderr: 1000K .......... .......... .......... .......... .......... 1% 8.77M 6s 2024-03-04T18:06:44.658 INFO:teuthology.orchestra.run.smithi137.stderr: 1050K .......... .......... .......... .......... .......... 1% 51.2M 6s 2024-03-04T18:06:44.658 INFO:teuthology.orchestra.run.smithi137.stderr: 1100K .......... .......... .......... .......... .......... 1% 79.4M 6s 2024-03-04T18:06:44.658 INFO:teuthology.orchestra.run.smithi137.stderr: 1150K .......... .......... .......... .......... .......... 1% 60.6M 6s 2024-03-04T18:06:44.658 INFO:teuthology.orchestra.run.smithi137.stderr: 1200K .......... .......... .......... .......... .......... 1% 83.0M 5s 2024-03-04T18:06:44.658 INFO:teuthology.orchestra.run.smithi137.stderr: 1250K .......... .......... .......... .......... .......... 1% 58.5M 5s 2024-03-04T18:06:44.658 INFO:teuthology.orchestra.run.smithi137.stderr: 1300K .......... .......... .......... .......... .......... 1% 82.2M 5s 2024-03-04T18:06:44.659 INFO:teuthology.orchestra.run.smithi137.stderr: 1350K .......... .......... .......... .......... .......... 1% 73.5M 5s 2024-03-04T18:06:44.659 INFO:teuthology.orchestra.run.smithi137.stderr: 1400K .......... .......... .......... .......... .......... 1% 82.7M 5s 2024-03-04T18:06:44.659 INFO:teuthology.orchestra.run.smithi137.stderr: 1450K .......... .......... .......... .......... .......... 1% 83.1M 5s 2024-03-04T18:06:46.074 INFO:teuthology.orchestra.run.smithi137.stderr: 1500K .......... .......... .......... .......... .......... 1% 86.9M 5s 2024-03-04T18:06:46.074 INFO:teuthology.orchestra.run.smithi137.stderr: 1550K .......... .......... .......... .......... .......... 1% 75.4M 4s 2024-03-04T18:06:46.074 INFO:teuthology.orchestra.run.smithi137.stderr: 1600K .......... .......... .......... .......... .......... 1% 86.4M 4s 2024-03-04T18:06:46.074 INFO:teuthology.orchestra.run.smithi137.stderr: 1650K .......... .......... .......... .......... .......... 1% 80.0M 4s 2024-03-04T18:06:46.074 INFO:teuthology.orchestra.run.smithi137.stderr: 1700K .......... .......... .......... .......... .......... 1% 75.9M 4s 2024-03-04T18:06:46.074 INFO:teuthology.orchestra.run.smithi137.stderr: 1750K .......... .......... .......... .......... .......... 1% 73.6M 4s 2024-03-04T18:06:46.074 INFO:teuthology.orchestra.run.smithi137.stderr: 1800K .......... .......... .......... .......... .......... 1% 79.3M 4s 2024-03-04T18:06:46.074 INFO:teuthology.orchestra.run.smithi137.stderr: 1850K .......... .......... .......... .......... .......... 2% 88.1M 4s 2024-03-04T18:06:46.074 INFO:teuthology.orchestra.run.smithi137.stderr: 1900K .......... .......... .......... .......... .......... 2% 90.5M 4s 2024-03-04T18:06:46.075 INFO:teuthology.orchestra.run.smithi137.stderr: 1950K .......... .......... .......... .......... .......... 2% 71.0M 4s 2024-03-04T18:06:46.075 INFO:teuthology.orchestra.run.smithi137.stderr: 2000K .......... .......... .......... .......... .......... 2% 26.5M 4s 2024-03-04T18:06:46.075 INFO:teuthology.orchestra.run.smithi137.stderr: 2050K .......... .......... .......... .......... .......... 2% 76.9M 4s 2024-03-04T18:06:46.075 INFO:teuthology.orchestra.run.smithi137.stderr: 2100K .......... .......... .......... .......... .......... 2% 78.6M 4s 2024-03-04T18:06:46.075 INFO:teuthology.orchestra.run.smithi137.stderr: 2150K .......... .......... .......... .......... .......... 2% 72.1M 4s 2024-03-04T18:06:46.075 INFO:teuthology.orchestra.run.smithi137.stderr: 2200K .......... .......... .......... .......... .......... 2% 75.6M 4s 2024-03-04T18:06:46.075 INFO:teuthology.orchestra.run.smithi137.stderr: 2250K .......... .......... .......... .......... .......... 2% 76.8M 3s 2024-03-04T18:06:46.075 INFO:teuthology.orchestra.run.smithi137.stderr: 2300K .......... .......... .......... .......... .......... 2% 78.0M 3s 2024-03-04T18:06:46.075 INFO:teuthology.orchestra.run.smithi137.stderr: 2350K .......... .......... .......... .......... .......... 2% 63.8M 3s 2024-03-04T18:06:46.075 INFO:teuthology.orchestra.run.smithi137.stderr: 2400K .......... .......... .......... .......... .......... 2% 79.0M 3s 2024-03-04T18:06:46.075 INFO:teuthology.orchestra.run.smithi137.stderr: 2450K .......... .......... .......... .......... .......... 2% 75.7M 3s 2024-03-04T18:06:46.076 INFO:teuthology.orchestra.run.smithi137.stderr: 2500K .......... .......... .......... .......... .......... 2% 76.2M 3s 2024-03-04T18:06:46.076 INFO:teuthology.orchestra.run.smithi137.stderr: 2550K .......... .......... .......... .......... .......... 2% 71.6M 3s 2024-03-04T18:06:46.076 INFO:teuthology.orchestra.run.smithi137.stderr: 2600K .......... .......... .......... .......... .......... 2% 80.9M 3s 2024-03-04T18:06:46.076 INFO:teuthology.orchestra.run.smithi137.stderr: 2650K .......... .......... .......... .......... .......... 2% 80.4M 3s 2024-03-04T18:06:46.076 INFO:teuthology.orchestra.run.smithi137.stderr: 2700K .......... .......... .......... .......... .......... 2% 79.3M 3s 2024-03-04T18:06:46.076 INFO:teuthology.orchestra.run.smithi137.stderr: 2750K .......... .......... .......... .......... .......... 3% 68.0M 3s 2024-03-04T18:06:46.076 INFO:teuthology.orchestra.run.smithi137.stderr: 2800K .......... .......... .......... .......... .......... 3% 74.5M 3s 2024-03-04T18:06:46.076 INFO:teuthology.orchestra.run.smithi137.stderr: 2850K .......... .......... .......... .......... .......... 3% 78.5M 3s 2024-03-04T18:06:46.076 INFO:teuthology.orchestra.run.smithi137.stderr: 2900K .......... .......... .......... .......... .......... 3% 78.6M 3s 2024-03-04T18:06:46.076 INFO:teuthology.orchestra.run.smithi137.stderr: 2950K .......... .......... .......... .......... .......... 3% 68.9M 3s 2024-03-04T18:06:46.077 INFO:teuthology.orchestra.run.smithi137.stderr: 3000K .......... .......... .......... .......... .......... 3% 77.2M 3s 2024-03-04T18:06:46.077 INFO:teuthology.orchestra.run.smithi137.stderr: 3050K .......... .......... .......... .......... .......... 3% 80.2M 3s 2024-03-04T18:06:46.077 INFO:teuthology.orchestra.run.smithi137.stderr: 3100K .......... .......... .......... .......... .......... 3% 81.1M 3s 2024-03-04T18:06:46.077 INFO:teuthology.orchestra.run.smithi137.stderr: 3150K .......... .......... .......... .......... .......... 3% 66.1M 3s 2024-03-04T18:06:46.077 INFO:teuthology.orchestra.run.smithi137.stderr: 3200K .......... .......... .......... .......... .......... 3% 79.9M 3s 2024-03-04T18:06:46.077 INFO:teuthology.orchestra.run.smithi137.stderr: 3250K .......... .......... .......... .......... .......... 3% 77.0M 3s 2024-03-04T18:06:46.077 INFO:teuthology.orchestra.run.smithi137.stderr: 3300K .......... .......... .......... .......... .......... 3% 83.3M 3s 2024-03-04T18:06:46.077 INFO:teuthology.orchestra.run.smithi137.stderr: 3350K .......... .......... .......... .......... .......... 3% 76.0M 3s 2024-03-04T18:06:46.077 INFO:teuthology.orchestra.run.smithi137.stderr: 3400K .......... .......... .......... .......... .......... 3% 85.3M 3s 2024-03-04T18:06:46.077 INFO:teuthology.orchestra.run.smithi137.stderr: 3450K .......... .......... .......... .......... .......... 3% 82.9M 3s 2024-03-04T18:06:46.077 INFO:teuthology.orchestra.run.smithi137.stderr: 3500K .......... .......... .......... .......... .......... 3% 80.9M 3s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 3550K .......... .......... .......... .......... .......... 3% 66.8M 3s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 3600K .......... .......... .......... .......... .......... 3% 79.8M 3s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 3650K .......... .......... .......... .......... .......... 3% 77.0M 3s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 3700K .......... .......... .......... .......... .......... 4% 78.9M 3s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 3750K .......... .......... .......... .......... .......... 4% 70.1M 3s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 3800K .......... .......... .......... .......... .......... 4% 79.0M 3s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 3850K .......... .......... .......... .......... .......... 4% 76.3M 2s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 3900K .......... .......... .......... .......... .......... 4% 77.3M 2s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 3950K .......... .......... .......... .......... .......... 4% 66.7M 2s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 4000K .......... .......... .......... .......... .......... 4% 76.1M 2s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 4050K .......... .......... .......... .......... .......... 4% 81.4M 2s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 4100K .......... .......... .......... .......... .......... 4% 74.4M 2s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 4150K .......... .......... .......... .......... .......... 4% 78.5M 2s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 4200K .......... .......... .......... .......... .......... 4% 72.6M 2s 2024-03-04T18:06:46.078 INFO:teuthology.orchestra.run.smithi137.stderr: 4250K .......... .......... .......... .......... .......... 4% 87.1M 2s 2024-03-04T18:06:46.079 INFO:teuthology.orchestra.run.smithi137.stderr: 4300K .......... .......... .......... .......... .......... 4% 94.8M 2s 2024-03-04T18:06:46.079 INFO:teuthology.orchestra.run.smithi137.stderr: 4350K .......... .......... .......... .......... .......... 4% 73.8M 2s 2024-03-04T18:06:46.079 INFO:teuthology.orchestra.run.smithi137.stderr: 4400K .......... .......... .......... .......... .......... 4% 80.9M 2s 2024-03-04T18:06:46.079 INFO:teuthology.orchestra.run.smithi137.stderr: 4450K .......... .......... .......... .......... .......... 4% 92.8M 2s 2024-03-04T18:06:46.079 INFO:teuthology.orchestra.run.smithi137.stderr: 4500K .......... .......... .......... .......... .......... 4% 92.5M 2s 2024-03-04T18:06:46.079 INFO:teuthology.orchestra.run.smithi137.stderr: 4550K .......... .......... .......... .......... .......... 4% 81.4M 2s 2024-03-04T18:06:46.079 INFO:teuthology.orchestra.run.smithi137.stderr: 4600K .......... .......... .......... .......... .......... 5% 91.0M 2s 2024-03-04T18:06:46.079 INFO:teuthology.orchestra.run.smithi137.stderr: 4650K .......... .......... .......... .......... .......... 5% 88.7M 2s 2024-03-04T18:06:46.079 INFO:teuthology.orchestra.run.smithi137.stderr: 4700K .......... .......... .......... .......... .......... 5% 85.8M 2s 2024-03-04T18:06:46.079 INFO:teuthology.orchestra.run.smithi137.stderr: 4750K .......... .......... .......... .......... .......... 5% 65.7M 2s 2024-03-04T18:06:46.079 INFO:teuthology.orchestra.run.smithi137.stderr: 4800K .......... .......... .......... .......... .......... 5% 133M 2s 2024-03-04T18:06:46.079 INFO:teuthology.orchestra.run.smithi137.stderr: 4850K .......... .......... .......... .......... .......... 5% 148M 2s 2024-03-04T18:06:46.079 INFO:teuthology.orchestra.run.smithi137.stderr: 4900K .......... .......... .......... .......... .......... 5% 169M 2s 2024-03-04T18:06:46.079 INFO:teuthology.orchestra.run.smithi137.stderr: 4950K .......... .......... .......... .......... .......... 5% 149M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5000K .......... .......... .......... .......... .......... 5% 159M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5050K .......... .......... .......... .......... .......... 5% 170M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5100K .......... .......... .......... .......... .......... 5% 171M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5150K .......... .......... .......... .......... .......... 5% 131M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5200K .......... .......... .......... .......... .......... 5% 172M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5250K .......... .......... .......... .......... .......... 5% 146M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5300K .......... .......... .......... .......... .......... 5% 152M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5350K .......... .......... .......... .......... .......... 5% 132M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5400K .......... .......... .......... .......... .......... 5% 148M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5450K .......... .......... .......... .......... .......... 5% 145M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5500K .......... .......... .......... .......... .......... 5% 162M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5550K .......... .......... .......... .......... .......... 6% 169M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5600K .......... .......... .......... .......... .......... 6% 167M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5650K .......... .......... .......... .......... .......... 6% 139M 2s 2024-03-04T18:06:46.080 INFO:teuthology.orchestra.run.smithi137.stderr: 5700K .......... .......... .......... .......... .......... 6% 169M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 5750K .......... .......... .......... .......... .......... 6% 166M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 5800K .......... .......... .......... .......... .......... 6% 168M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 5850K .......... .......... .......... .......... .......... 6% 156M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 5900K .......... .......... .......... .......... .......... 6% 172M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 5950K .......... .......... .......... .......... .......... 6% 177M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 6000K .......... .......... .......... .......... .......... 6% 155M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 6050K .......... .......... .......... .......... .......... 6% 143M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 6100K .......... .......... .......... .......... .......... 6% 169M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 6150K .......... .......... .......... .......... .......... 6% 150M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 6200K .......... .......... .......... .......... .......... 6% 169M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 6250K .......... .......... .......... .......... .......... 6% 152M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 6300K .......... .......... .......... .......... .......... 6% 165M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 6350K .......... .......... .......... .......... .......... 6% 170M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 6400K .......... .......... .......... .......... .......... 6% 171M 2s 2024-03-04T18:06:46.081 INFO:teuthology.orchestra.run.smithi137.stderr: 6450K .......... .......... .......... .......... .......... 6% 134M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 6500K .......... .......... .......... .......... .......... 7% 172M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 6550K .......... .......... .......... .......... .......... 7% 175M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 6600K .......... .......... .......... .......... .......... 7% 169M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 6650K .......... .......... .......... .......... .......... 7% 144M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 6700K .......... .......... .......... .......... .......... 7% 161M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 6750K .......... .......... .......... .......... .......... 7% 186M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 6800K .......... .......... .......... .......... .......... 7% 179M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 6850K .......... .......... .......... .......... .......... 7% 149M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 6900K .......... .......... .......... .......... .......... 7% 186M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 6950K .......... .......... .......... .......... .......... 7% 183M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 7000K .......... .......... .......... .......... .......... 7% 202M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 7050K .......... .......... .......... .......... .......... 7% 181M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 7100K .......... .......... .......... .......... .......... 7% 181M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 7150K .......... .......... .......... .......... .......... 7% 179M 2s 2024-03-04T18:06:46.082 INFO:teuthology.orchestra.run.smithi137.stderr: 7200K .......... .......... .......... .......... .......... 7% 188M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7250K .......... .......... .......... .......... .......... 7% 118M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7300K .......... .......... .......... .......... .......... 7% 119M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7350K .......... .......... .......... .......... .......... 7% 160M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7400K .......... .......... .......... .......... .......... 8% 223M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7450K .......... .......... .......... .......... .......... 8% 121M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7500K .......... .......... .......... .......... .......... 8% 111M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7550K .......... .......... .......... .......... .......... 8% 101M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7600K .......... .......... .......... .......... .......... 8% 118M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7650K .......... .......... .......... .......... .......... 8% 82.5M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7700K .......... .......... .......... .......... .......... 8% 111M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7750K .......... .......... .......... .......... .......... 8% 116M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7800K .......... .......... .......... .......... .......... 8% 110M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7850K .......... .......... .......... .......... .......... 8% 110M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7900K .......... .......... .......... .......... .......... 8% 111M 2s 2024-03-04T18:06:46.083 INFO:teuthology.orchestra.run.smithi137.stderr: 7950K .......... .......... .......... .......... .......... 8% 112M 1s 2024-03-04T18:06:46.084 INFO:teuthology.orchestra.run.smithi137.stderr: 8000K .......... .......... .......... .......... .......... 8% 105M 1s 2024-03-04T18:06:46.084 INFO:teuthology.orchestra.run.smithi137.stderr: 8050K .......... .......... .......... .......... .......... 8% 95.1M 1s 2024-03-04T18:06:46.084 INFO:teuthology.orchestra.run.smithi137.stderr: 8100K .......... .......... .......... .......... .......... 8% 118M 1s 2024-03-04T18:06:46.084 INFO:teuthology.orchestra.run.smithi137.stderr: 8150K .......... .......... .......... .......... .......... 8% 107M 1s 2024-03-04T18:06:46.084 INFO:teuthology.orchestra.run.smithi137.stderr: 8200K .......... .......... .......... .......... .......... 8% 115M 1s 2024-03-04T18:06:46.084 INFO:teuthology.orchestra.run.smithi137.stderr: 8250K .......... .......... .......... .......... .......... 8% 104M 1s 2024-03-04T18:06:46.084 INFO:teuthology.orchestra.run.smithi137.stderr: 8300K .......... .......... .......... .......... .......... 8% 96.6M 1s 2024-03-04T18:06:46.084 INFO:teuthology.orchestra.run.smithi137.stderr: 8350K .......... .......... .......... .......... .......... 9% 140M 1s 2024-03-04T18:06:46.084 INFO:teuthology.orchestra.run.smithi137.stderr: 8400K .......... .......... .......... .......... .......... 9% 123M 1s 2024-03-04T18:06:46.084 INFO:teuthology.orchestra.run.smithi137.stderr: 8450K .......... .......... .......... .......... .......... 9% 88.0M 1s 2024-03-04T18:06:46.084 INFO:teuthology.orchestra.run.smithi137.stderr: 8500K .......... .......... .......... .......... .......... 9% 119M 1s 2024-03-04T18:06:46.084 INFO:teuthology.orchestra.run.smithi137.stderr: 8550K .......... .......... .......... .......... .......... 9% 113M 1s 2024-03-04T18:06:46.084 INFO:teuthology.orchestra.run.smithi137.stderr: 8600K .......... .......... .......... .......... .......... 9% 113M 1s 2024-03-04T18:06:46.084 INFO:teuthology.orchestra.run.smithi137.stderr: 8650K .......... .......... .......... .......... .......... 9% 105M 1s 2024-03-04T18:06:46.085 INFO:teuthology.orchestra.run.smithi137.stderr: 8700K .......... .......... .......... .......... .......... 9% 109M 1s 2024-03-04T18:06:46.085 INFO:teuthology.orchestra.run.smithi137.stderr: 8750K .......... .......... .......... .......... .......... 9% 124M 1s 2024-03-04T18:06:46.085 INFO:teuthology.orchestra.run.smithi137.stderr: 8800K .......... .......... .......... .......... .......... 9% 113M 1s 2024-03-04T18:06:46.085 INFO:teuthology.orchestra.run.smithi137.stderr: 8850K .......... .......... .......... .......... .......... 9% 90.0M 1s 2024-03-04T18:06:46.085 INFO:teuthology.orchestra.run.smithi137.stderr: 8900K .......... .......... .......... .......... .......... 9% 114M 1s 2024-03-04T18:06:46.085 INFO:teuthology.orchestra.run.smithi137.stderr: 8950K .......... .......... .......... .......... .......... 9% 113M 1s 2024-03-04T18:06:46.085 INFO:teuthology.orchestra.run.smithi137.stderr: 9000K .......... .......... .......... .......... .......... 9% 115M 1s 2024-03-04T18:06:46.085 INFO:teuthology.orchestra.run.smithi137.stderr: 9050K .......... .......... .......... .......... .......... 9% 104M 1s 2024-03-04T18:06:46.085 INFO:teuthology.orchestra.run.smithi137.stderr: 9100K .......... .......... .......... .......... .......... 9% 107M 1s 2024-03-04T18:06:46.085 INFO:teuthology.orchestra.run.smithi137.stderr: 9150K .......... .......... .......... .......... .......... 9% 105M 1s 2024-03-04T18:06:46.085 INFO:teuthology.orchestra.run.smithi137.stderr: 9200K .......... .......... .......... .......... .......... 9% 121M 1s 2024-03-04T18:06:46.085 INFO:teuthology.orchestra.run.smithi137.stderr: 9250K .......... .......... .......... .......... .......... 10% 91.6M 1s 2024-03-04T18:06:46.085 INFO:teuthology.orchestra.run.smithi137.stderr: 9300K .......... .......... .......... .......... .......... 10% 116M 1s 2024-03-04T18:06:46.085 INFO:teuthology.orchestra.run.smithi137.stderr: 9350K .......... .......... .......... .......... .......... 10% 114M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 9400K .......... .......... .......... .......... .......... 10% 112M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 9450K .......... .......... .......... .......... .......... 10% 106M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 9500K .......... .......... .......... .......... .......... 10% 120M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 9550K .......... .......... .......... .......... .......... 10% 111M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 9600K .......... .......... .......... .......... .......... 10% 112M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 9650K .......... .......... .......... .......... .......... 10% 92.6M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 9700K .......... .......... .......... .......... .......... 10% 114M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 9750K .......... .......... .......... .......... .......... 10% 115M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 9800K .......... .......... .......... .......... .......... 10% 111M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 9850K .......... .......... .......... .......... .......... 10% 98.3M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 9900K .......... .......... .......... .......... .......... 10% 111M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 9950K .......... .......... .......... .......... .......... 10% 116M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 10000K .......... .......... .......... .......... .......... 10% 114M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 10050K .......... .......... .......... .......... .......... 10% 90.4M 1s 2024-03-04T18:06:46.086 INFO:teuthology.orchestra.run.smithi137.stderr: 10100K .......... .......... .......... .......... .......... 10% 106M 1s 2024-03-04T18:06:46.087 INFO:teuthology.orchestra.run.smithi137.stderr: 10150K .......... .......... .......... .......... .......... 10% 109M 1s 2024-03-04T18:06:46.087 INFO:teuthology.orchestra.run.smithi137.stderr: 10200K .......... .......... .......... .......... .......... 11% 117M 1s 2024-03-04T18:06:46.087 INFO:teuthology.orchestra.run.smithi137.stderr: 10250K .......... .......... .......... .......... .......... 11% 98.9M 1s 2024-03-04T18:06:46.087 INFO:teuthology.orchestra.run.smithi137.stderr: 10300K .......... .......... .......... .......... .......... 11% 109M 1s 2024-03-04T18:06:46.087 INFO:teuthology.orchestra.run.smithi137.stderr: 10350K .......... .......... .......... .......... .......... 11% 119M 1s 2024-03-04T18:06:46.087 INFO:teuthology.orchestra.run.smithi137.stderr: 10400K .......... .......... .......... .......... .......... 11% 115M 1s 2024-03-04T18:06:46.087 INFO:teuthology.orchestra.run.smithi137.stderr: 10450K .......... .......... .......... .......... .......... 11% 79.1M 1s 2024-03-04T18:06:46.087 INFO:teuthology.orchestra.run.smithi137.stderr: 10500K .......... .......... .......... .......... .......... 11% 111M 1s 2024-03-04T18:06:46.087 INFO:teuthology.orchestra.run.smithi137.stderr: 10550K .......... .......... .......... .......... .......... 11% 130M 1s 2024-03-04T18:06:46.087 INFO:teuthology.orchestra.run.smithi137.stderr: 10600K .......... .......... .......... .......... .......... 11% 96.4M 1s 2024-03-04T18:06:46.087 INFO:teuthology.orchestra.run.smithi137.stderr: 10650K .......... .......... .......... .......... .......... 11% 117M 1s 2024-03-04T18:06:46.087 INFO:teuthology.orchestra.run.smithi137.stderr: 10700K .......... .......... .......... .......... .......... 11% 94.2M 1s 2024-03-04T18:06:46.087 INFO:teuthology.orchestra.run.smithi137.stderr: 10750K .......... .......... .......... .......... .......... 11% 126M 1s 2024-03-04T18:06:46.087 INFO:teuthology.orchestra.run.smithi137.stderr: 10800K .......... .......... .......... .......... .......... 11% 126M 1s 2024-03-04T18:06:46.088 INFO:teuthology.orchestra.run.smithi137.stderr: 10850K .......... .......... .......... .......... .......... 11% 65.6M 1s 2024-03-04T18:06:46.088 INFO:teuthology.orchestra.run.smithi137.stderr: 10900K .......... .......... .......... .......... .......... 11% 136M 1s 2024-03-04T18:06:46.088 INFO:teuthology.orchestra.run.smithi137.stderr: 10950K .......... .......... .......... .......... .......... 11% 126M 1s 2024-03-04T18:06:46.088 INFO:teuthology.orchestra.run.smithi137.stderr: 11000K .......... .......... .......... .......... .......... 11% 109M 1s 2024-03-04T18:06:46.088 INFO:teuthology.orchestra.run.smithi137.stderr: 11050K .......... .......... .......... .......... .......... 11% 108M 1s 2024-03-04T18:06:46.088 INFO:teuthology.orchestra.run.smithi137.stderr: 11100K .......... .......... .......... .......... .......... 12% 103M 1s 2024-03-04T18:06:46.088 INFO:teuthology.orchestra.run.smithi137.stderr: 11150K .......... .......... .......... .......... .......... 12% 116M 1s 2024-03-04T18:06:46.088 INFO:teuthology.orchestra.run.smithi137.stderr: 11200K .......... .......... .......... .......... .......... 12% 109M 1s 2024-03-04T18:06:46.088 INFO:teuthology.orchestra.run.smithi137.stderr: 11250K .......... .......... .......... .......... .......... 12% 92.6M 1s 2024-03-04T18:06:46.088 INFO:teuthology.orchestra.run.smithi137.stderr: 11300K .......... .......... .......... .......... .......... 12% 114M 1s 2024-03-04T18:06:46.088 INFO:teuthology.orchestra.run.smithi137.stderr: 11350K .......... .......... .......... .......... .......... 12% 105M 1s 2024-03-04T18:06:46.088 INFO:teuthology.orchestra.run.smithi137.stderr: 11400K .......... .......... .......... .......... .......... 12% 116M 1s 2024-03-04T18:06:46.088 INFO:teuthology.orchestra.run.smithi137.stderr: 11450K .......... .......... .......... .......... .......... 12% 105M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 11500K .......... .......... .......... .......... .......... 12% 113M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 11550K .......... .......... .......... .......... .......... 12% 81.1M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 11600K .......... .......... .......... .......... .......... 12% 121M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 11650K .......... .......... .......... .......... .......... 12% 73.1M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 11700K .......... .......... .......... .......... .......... 12% 82.1M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 11750K .......... .......... .......... .......... .......... 12% 106M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 11800K .......... .......... .......... .......... .......... 12% 57.3M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 11850K .......... .......... .......... .......... .......... 12% 141M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 11900K .......... .......... .......... .......... .......... 12% 60.6M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 11950K .......... .......... .......... .......... .......... 12% 59.6M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 12000K .......... .......... .......... .......... .......... 12% 126M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 12050K .......... .......... .......... .......... .......... 13% 86.1M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 12100K .......... .......... .......... .......... .......... 13% 118M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 12150K .......... .......... .......... .......... .......... 13% 105M 1s 2024-03-04T18:06:46.089 INFO:teuthology.orchestra.run.smithi137.stderr: 12200K .......... .......... .......... .......... .......... 13% 109M 1s 2024-03-04T18:06:46.090 INFO:teuthology.orchestra.run.smithi137.stderr: 12250K .......... .......... .......... .......... .......... 13% 109M 1s 2024-03-04T18:06:46.090 INFO:teuthology.orchestra.run.smithi137.stderr: 12300K .......... .......... .......... .......... .......... 13% 109M 1s 2024-03-04T18:06:46.090 INFO:teuthology.orchestra.run.smithi137.stderr: 12350K .......... .......... .......... .......... .......... 13% 104M 1s 2024-03-04T18:06:46.090 INFO:teuthology.orchestra.run.smithi137.stderr: 12400K .......... .......... .......... .......... .......... 13% 114M 1s 2024-03-04T18:06:46.090 INFO:teuthology.orchestra.run.smithi137.stderr: 12450K .......... .......... .......... .......... .......... 13% 96.8M 1s 2024-03-04T18:06:46.090 INFO:teuthology.orchestra.run.smithi137.stderr: 12500K .......... .......... .......... .......... .......... 13% 108M 1s 2024-03-04T18:06:46.090 INFO:teuthology.orchestra.run.smithi137.stderr: 12550K .......... .......... .......... .......... .......... 13% 117M 1s 2024-03-04T18:06:46.090 INFO:teuthology.orchestra.run.smithi137.stderr: 12600K .......... .......... .......... .......... .......... 13% 114M 1s 2024-03-04T18:06:46.090 INFO:teuthology.orchestra.run.smithi137.stderr: 12650K .......... .......... .......... .......... .......... 13% 94.2M 1s 2024-03-04T18:06:46.090 INFO:teuthology.orchestra.run.smithi137.stderr: 12700K .......... .......... .......... .......... .......... 13% 114M 1s 2024-03-04T18:06:46.090 INFO:teuthology.orchestra.run.smithi137.stderr: 12750K .......... .......... .......... .......... .......... 13% 104M 1s 2024-03-04T18:06:46.090 INFO:teuthology.orchestra.run.smithi137.stderr: 12800K .......... .......... .......... .......... .......... 13% 134M 1s 2024-03-04T18:06:46.090 INFO:teuthology.orchestra.run.smithi137.stderr: 12850K .......... .......... .......... .......... .......... 13% 85.6M 1s 2024-03-04T18:06:46.090 INFO:teuthology.orchestra.run.smithi137.stderr: 12900K .......... .......... .......... .......... .......... 13% 121M 1s 2024-03-04T18:06:46.091 INFO:teuthology.orchestra.run.smithi137.stderr: 12950K .......... .......... .......... .......... .......... 13% 112M 1s 2024-03-04T18:06:46.091 INFO:teuthology.orchestra.run.smithi137.stderr: 13000K .......... .......... .......... .......... .......... 14% 121M 1s 2024-03-04T18:06:46.091 INFO:teuthology.orchestra.run.smithi137.stderr: 13050K .......... .......... .......... .......... .......... 14% 103M 1s 2024-03-04T18:06:46.091 INFO:teuthology.orchestra.run.smithi137.stderr: 13100K .......... .......... .......... .......... .......... 14% 112M 1s 2024-03-04T18:06:46.091 INFO:teuthology.orchestra.run.smithi137.stderr: 13150K .......... .......... .......... .......... .......... 14% 103M 1s 2024-03-04T18:06:46.091 INFO:teuthology.orchestra.run.smithi137.stderr: 13200K .......... .......... .......... .......... .......... 14% 111M 1s 2024-03-04T18:06:46.091 INFO:teuthology.orchestra.run.smithi137.stderr: 13250K .......... .......... .......... .......... .......... 14% 89.1M 1s 2024-03-04T18:06:46.091 INFO:teuthology.orchestra.run.smithi137.stderr: 13300K .......... .......... .......... .......... .......... 14% 104M 1s 2024-03-04T18:06:46.091 INFO:teuthology.orchestra.run.smithi137.stderr: 13350K .......... .......... .......... .......... .......... 14% 74.2M 1s 2024-03-04T18:06:46.091 INFO:teuthology.orchestra.run.smithi137.stderr: 13400K .......... .......... .......... .......... .......... 14% 110M 1s 2024-03-04T18:06:46.091 INFO:teuthology.orchestra.run.smithi137.stderr: 13450K .......... .......... .......... .......... .......... 14% 110M 1s 2024-03-04T18:06:46.091 INFO:teuthology.orchestra.run.smithi137.stderr: 13500K .......... .......... .......... .......... .......... 14% 110M 1s 2024-03-04T18:06:46.091 INFO:teuthology.orchestra.run.smithi137.stderr: 13550K .......... .......... .......... .......... .......... 14% 104M 1s 2024-03-04T18:06:46.091 INFO:teuthology.orchestra.run.smithi137.stderr: 13600K .......... .......... .......... .......... .......... 14% 110M 1s 2024-03-04T18:06:46.092 INFO:teuthology.orchestra.run.smithi137.stderr: 13650K .......... .......... .......... .......... .......... 14% 95.0M 1s 2024-03-04T18:06:46.092 INFO:teuthology.orchestra.run.smithi137.stderr: 13700K .......... .......... .......... .......... .......... 14% 115M 1s 2024-03-04T18:06:46.092 INFO:teuthology.orchestra.run.smithi137.stderr: 13750K .......... .......... .......... .......... .......... 14% 110M 1s 2024-03-04T18:06:46.092 INFO:teuthology.orchestra.run.smithi137.stderr: 13800K .......... .......... .......... .......... .......... 14% 115M 1s 2024-03-04T18:06:46.092 INFO:teuthology.orchestra.run.smithi137.stderr: 13850K .......... .......... .......... .......... .......... 14% 100M 1s 2024-03-04T18:06:46.092 INFO:teuthology.orchestra.run.smithi137.stderr: 13900K .......... .......... .......... .......... .......... 15% 108M 1s 2024-03-04T18:06:46.092 INFO:teuthology.orchestra.run.smithi137.stderr: 13950K .......... .......... .......... .......... .......... 15% 121M 1s 2024-03-04T18:06:46.092 INFO:teuthology.orchestra.run.smithi137.stderr: 14000K .......... .......... .......... .......... .......... 15% 97.3M 1s 2024-03-04T18:06:46.092 INFO:teuthology.orchestra.run.smithi137.stderr: 14050K .......... .......... .......... .......... .......... 15% 99.1M 1s 2024-03-04T18:06:46.092 INFO:teuthology.orchestra.run.smithi137.stderr: 14100K .......... .......... .......... .......... .......... 15% 122M 1s 2024-03-04T18:06:46.092 INFO:teuthology.orchestra.run.smithi137.stderr: 14150K .......... .......... .......... .......... .......... 15% 113M 1s 2024-03-04T18:06:46.092 INFO:teuthology.orchestra.run.smithi137.stderr: 14200K .......... .......... .......... .......... .......... 15% 111M 1s 2024-03-04T18:06:46.092 INFO:teuthology.orchestra.run.smithi137.stderr: 14250K .......... .......... .......... .......... .......... 15% 104M 1s 2024-03-04T18:06:46.092 INFO:teuthology.orchestra.run.smithi137.stderr: 14300K .......... .......... .......... .......... .......... 15% 116M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 14350K .......... .......... .......... .......... .......... 15% 108M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 14400K .......... .......... .......... .......... .......... 15% 115M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 14450K .......... .......... .......... .......... .......... 15% 88.4M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 14500K .......... .......... .......... .......... .......... 15% 105M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 14550K .......... .......... .......... .......... .......... 15% 115M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 14600K .......... .......... .......... .......... .......... 15% 112M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 14650K .......... .......... .......... .......... .......... 15% 107M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 14700K .......... .......... .......... .......... .......... 15% 114M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 14750K .......... .......... .......... .......... .......... 15% 101M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 14800K .......... .......... .......... .......... .......... 15% 123M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 14850K .......... .......... .......... .......... .......... 16% 86.1M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 14900K .......... .......... .......... .......... .......... 16% 118M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 14950K .......... .......... .......... .......... .......... 16% 105M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 15000K .......... .......... .......... .......... .......... 16% 88.6M 1s 2024-03-04T18:06:46.093 INFO:teuthology.orchestra.run.smithi137.stderr: 15050K .......... .......... .......... .......... .......... 16% 42.3M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15100K .......... .......... .......... .......... .......... 16% 131M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15150K .......... .......... .......... .......... .......... 16% 110M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15200K .......... .......... .......... .......... .......... 16% 47.4M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15250K .......... .......... .......... .......... .......... 16% 91.9M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15300K .......... .......... .......... .......... .......... 16% 114M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15350K .......... .......... .......... .......... .......... 16% 111M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15400K .......... .......... .......... .......... .......... 16% 113M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15450K .......... .......... .......... .......... .......... 16% 111M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15500K .......... .......... .......... .......... .......... 16% 99.4M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15550K .......... .......... .......... .......... .......... 16% 117M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15600K .......... .......... .......... .......... .......... 16% 111M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15650K .......... .......... .......... .......... .......... 16% 90.9M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15700K .......... .......... .......... .......... .......... 16% 114M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15750K .......... .......... .......... .......... .......... 17% 112M 1s 2024-03-04T18:06:46.094 INFO:teuthology.orchestra.run.smithi137.stderr: 15800K .......... .......... .......... .......... .......... 17% 111M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 15850K .......... .......... .......... .......... .......... 17% 108M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 15900K .......... .......... .......... .......... .......... 17% 114M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 15950K .......... .......... .......... .......... .......... 17% 116M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 16000K .......... .......... .......... .......... .......... 17% 109M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 16050K .......... .......... .......... .......... .......... 17% 91.9M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 16100K .......... .......... .......... .......... .......... 17% 51.5M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 16150K .......... .......... .......... .......... .......... 17% 105M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 16200K .......... .......... .......... .......... .......... 17% 112M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 16250K .......... .......... .......... .......... .......... 17% 92.4M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 16300K .......... .......... .......... .......... .......... 17% 113M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 16350K .......... .......... .......... .......... .......... 17% 119M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 16400K .......... .......... .......... .......... .......... 17% 109M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 16450K .......... .......... .......... .......... .......... 17% 93.1M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 16500K .......... .......... .......... .......... .......... 17% 97.1M 1s 2024-03-04T18:06:46.095 INFO:teuthology.orchestra.run.smithi137.stderr: 16550K .......... .......... .......... .......... .......... 17% 105M 1s 2024-03-04T18:06:46.096 INFO:teuthology.orchestra.run.smithi137.stderr: 16600K .......... .......... .......... .......... .......... 17% 128M 1s 2024-03-04T18:06:46.096 INFO:teuthology.orchestra.run.smithi137.stderr: 16650K .......... .......... .......... .......... .......... 17% 110M 1s 2024-03-04T18:06:46.096 INFO:teuthology.orchestra.run.smithi137.stderr: 16700K .......... .......... .......... .......... .......... 18% 102M 1s 2024-03-04T18:06:46.096 INFO:teuthology.orchestra.run.smithi137.stderr: 16750K .......... .......... .......... .......... .......... 18% 115M 1s 2024-03-04T18:06:46.096 INFO:teuthology.orchestra.run.smithi137.stderr: 16800K .......... .......... .......... .......... .......... 18% 110M 1s 2024-03-04T18:06:46.096 INFO:teuthology.orchestra.run.smithi137.stderr: 16850K .......... .......... .......... .......... .......... 18% 91.9M 1s 2024-03-04T18:06:46.096 INFO:teuthology.orchestra.run.smithi137.stderr: 16900K .......... .......... .......... .......... .......... 18% 113M 1s 2024-03-04T18:06:46.096 INFO:teuthology.orchestra.run.smithi137.stderr: 16950K .......... .......... .......... .......... .......... 18% 108M 1s 2024-03-04T18:06:46.096 INFO:teuthology.orchestra.run.smithi137.stderr: 17000K .......... .......... .......... .......... .......... 18% 118M 1s 2024-03-04T18:06:46.096 INFO:teuthology.orchestra.run.smithi137.stderr: 17050K .......... .......... .......... .......... .......... 18% 101M 1s 2024-03-04T18:06:46.096 INFO:teuthology.orchestra.run.smithi137.stderr: 17100K .......... .......... .......... .......... .......... 18% 106M 1s 2024-03-04T18:06:46.096 INFO:teuthology.orchestra.run.smithi137.stderr: 17150K .......... .......... .......... .......... .......... 18% 114M 1s 2024-03-04T18:06:46.096 INFO:teuthology.orchestra.run.smithi137.stderr: 17200K .......... .......... .......... .......... .......... 18% 112M 1s 2024-03-04T18:06:46.096 INFO:teuthology.orchestra.run.smithi137.stderr: 17250K .......... .......... .......... .......... .......... 18% 92.2M 1s 2024-03-04T18:06:46.097 INFO:teuthology.orchestra.run.smithi137.stderr: 17300K .......... .......... .......... .......... .......... 18% 108M 1s 2024-03-04T18:06:46.097 INFO:teuthology.orchestra.run.smithi137.stderr: 17350K .......... .......... .......... .......... .......... 18% 112M 1s 2024-03-04T18:06:46.097 INFO:teuthology.orchestra.run.smithi137.stderr: 17400K .......... .......... .......... .......... .......... 18% 117M 1s 2024-03-04T18:06:46.097 INFO:teuthology.orchestra.run.smithi137.stderr: 17450K .......... .......... .......... .......... .......... 18% 108M 1s 2024-03-04T18:06:46.097 INFO:teuthology.orchestra.run.smithi137.stderr: 17500K .......... .......... .......... .......... .......... 18% 113M 1s 2024-03-04T18:06:46.097 INFO:teuthology.orchestra.run.smithi137.stderr: 17550K .......... .......... .......... .......... .......... 18% 111M 1s 2024-03-04T18:06:46.097 INFO:teuthology.orchestra.run.smithi137.stderr: 17600K .......... .......... .......... .......... .......... 18% 112M 1s 2024-03-04T18:06:46.097 INFO:teuthology.orchestra.run.smithi137.stderr: 17650K .......... .......... .......... .......... .......... 19% 83.1M 1s 2024-03-04T18:06:46.097 INFO:teuthology.orchestra.run.smithi137.stderr: 17700K .......... .......... .......... .......... .......... 19% 110M 1s 2024-03-04T18:06:46.097 INFO:teuthology.orchestra.run.smithi137.stderr: 17750K .......... .......... .......... .......... .......... 19% 105M 1s 2024-03-04T18:06:46.098 INFO:teuthology.orchestra.run.smithi137.stderr: 17800K .......... .......... .......... .......... .......... 19% 105M 1s 2024-03-04T18:06:46.098 INFO:teuthology.orchestra.run.smithi137.stderr: 17850K .......... .......... .......... .......... .......... 19% 96.5M 1s 2024-03-04T18:06:46.098 INFO:teuthology.orchestra.run.smithi137.stderr: 17900K .......... .......... .......... .......... .......... 19% 121M 1s 2024-03-04T18:06:46.098 INFO:teuthology.orchestra.run.smithi137.stderr: 17950K .......... .......... .......... .......... .......... 19% 102M 1s 2024-03-04T18:06:46.098 INFO:teuthology.orchestra.run.smithi137.stderr: 18000K .......... .......... .......... .......... .......... 19% 108M 1s 2024-03-04T18:06:46.098 INFO:teuthology.orchestra.run.smithi137.stderr: 18050K .......... .......... .......... .......... .......... 19% 84.8M 1s 2024-03-04T18:06:46.098 INFO:teuthology.orchestra.run.smithi137.stderr: 18100K .......... .......... .......... .......... .......... 19% 88.6M 1s 2024-03-04T18:06:46.098 INFO:teuthology.orchestra.run.smithi137.stderr: 18150K .......... .......... .......... .......... .......... 19% 137M 1s 2024-03-04T18:06:46.098 INFO:teuthology.orchestra.run.smithi137.stderr: 18200K .......... .......... .......... .......... .......... 19% 95.9M 1s 2024-03-04T18:06:46.098 INFO:teuthology.orchestra.run.smithi137.stderr: 18250K .......... .......... .......... .......... .......... 19% 117M 1s 2024-03-04T18:06:46.098 INFO:teuthology.orchestra.run.smithi137.stderr: 18300K .......... .......... .......... .......... .......... 19% 97.3M 1s 2024-03-04T18:06:46.098 INFO:teuthology.orchestra.run.smithi137.stderr: 18350K .......... .......... .......... .......... .......... 19% 136M 1s 2024-03-04T18:06:46.098 INFO:teuthology.orchestra.run.smithi137.stderr: 18400K .......... .......... .......... .......... .......... 19% 114M 1s 2024-03-04T18:06:46.099 INFO:teuthology.orchestra.run.smithi137.stderr: 18450K .......... .......... .......... .......... .......... 19% 89.4M 1s 2024-03-04T18:06:46.099 INFO:teuthology.orchestra.run.smithi137.stderr: 18500K .......... .......... .......... .......... .......... 19% 100M 1s 2024-03-04T18:06:46.099 INFO:teuthology.orchestra.run.smithi137.stderr: 18550K .......... .......... .......... .......... .......... 20% 105M 1s 2024-03-04T18:06:46.099 INFO:teuthology.orchestra.run.smithi137.stderr: 18600K .......... .......... .......... .......... .......... 20% 124M 1s 2024-03-04T18:06:46.099 INFO:teuthology.orchestra.run.smithi137.stderr: 18650K .......... .......... .......... .......... .......... 20% 95.1M 1s 2024-03-04T18:06:46.099 INFO:teuthology.orchestra.run.smithi137.stderr: 18700K .......... .......... .......... .......... .......... 20% 113M 1s 2024-03-04T18:06:46.099 INFO:teuthology.orchestra.run.smithi137.stderr: 18750K .......... .......... .......... .......... .......... 20% 126M 1s 2024-03-04T18:06:46.099 INFO:teuthology.orchestra.run.smithi137.stderr: 18800K .......... .......... .......... .......... .......... 20% 118M 1s 2024-03-04T18:06:46.099 INFO:teuthology.orchestra.run.smithi137.stderr: 18850K .......... .......... .......... .......... .......... 20% 86.1M 1s 2024-03-04T18:06:46.099 INFO:teuthology.orchestra.run.smithi137.stderr: 18900K .......... .......... .......... .......... .......... 20% 112M 1s 2024-03-04T18:06:46.099 INFO:teuthology.orchestra.run.smithi137.stderr: 18950K .......... .......... .......... .......... .......... 20% 117M 1s 2024-03-04T18:06:46.099 INFO:teuthology.orchestra.run.smithi137.stderr: 19000K .......... .......... .......... .......... .......... 20% 109M 1s 2024-03-04T18:06:46.099 INFO:teuthology.orchestra.run.smithi137.stderr: 19050K .......... .......... .......... .......... .......... 20% 105M 1s 2024-03-04T18:06:46.099 INFO:teuthology.orchestra.run.smithi137.stderr: 19100K .......... .......... .......... .......... .......... 20% 112M 1s 2024-03-04T18:06:46.100 INFO:teuthology.orchestra.run.smithi137.stderr: 19150K .......... .......... .......... .......... .......... 20% 107M 1s 2024-03-04T18:06:46.100 INFO:teuthology.orchestra.run.smithi137.stderr: 19200K .......... .......... .......... .......... .......... 20% 86.7M 1s 2024-03-04T18:06:46.100 INFO:teuthology.orchestra.run.smithi137.stderr: 19250K .......... .......... .......... .......... .......... 20% 87.8M 1s 2024-03-04T18:06:46.100 INFO:teuthology.orchestra.run.smithi137.stderr: 19300K .......... .......... .......... .......... .......... 20% 118M 1s 2024-03-04T18:06:46.100 INFO:teuthology.orchestra.run.smithi137.stderr: 19350K .......... .......... .......... .......... .......... 20% 114M 1s 2024-03-04T18:06:46.100 INFO:teuthology.orchestra.run.smithi137.stderr: 19400K .......... .......... .......... .......... .......... 20% 113M 1s 2024-03-04T18:06:46.100 INFO:teuthology.orchestra.run.smithi137.stderr: 19450K .......... .......... .......... .......... .......... 20% 98.0M 1s 2024-03-04T18:06:46.100 INFO:teuthology.orchestra.run.smithi137.stderr: 19500K .......... .......... .......... .......... .......... 21% 113M 1s 2024-03-04T18:06:46.100 INFO:teuthology.orchestra.run.smithi137.stderr: 19550K .......... .......... .......... .......... .......... 21% 116M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 19600K .......... .......... .......... .......... .......... 21% 113M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 19650K .......... .......... .......... .......... .......... 21% 87.8M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 19700K .......... .......... .......... .......... .......... 21% 110M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 19750K .......... .......... .......... .......... .......... 21% 115M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 19800K .......... .......... .......... .......... .......... 21% 112M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 19850K .......... .......... .......... .......... .......... 21% 98.9M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 19900K .......... .......... .......... .......... .......... 21% 101M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 19950K .......... .......... .......... .......... .......... 21% 116M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 20000K .......... .......... .......... .......... .......... 21% 104M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 20050K .......... .......... .......... .......... .......... 21% 100M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 20100K .......... .......... .......... .......... .......... 21% 116M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 20150K .......... .......... .......... .......... .......... 21% 100M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 20200K .......... .......... .......... .......... .......... 21% 126M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 20250K .......... .......... .......... .......... .......... 21% 98.5M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 20300K .......... .......... .......... .......... .......... 21% 116M 1s 2024-03-04T18:06:46.101 INFO:teuthology.orchestra.run.smithi137.stderr: 20350K .......... .......... .......... .......... .......... 21% 104M 1s 2024-03-04T18:06:46.102 INFO:teuthology.orchestra.run.smithi137.stderr: 20400K .......... .......... .......... .......... .......... 22% 123M 1s 2024-03-04T18:06:46.102 INFO:teuthology.orchestra.run.smithi137.stderr: 20450K .......... .......... .......... .......... .......... 22% 92.5M 1s 2024-03-04T18:06:46.102 INFO:teuthology.orchestra.run.smithi137.stderr: 20500K .......... .......... .......... .......... .......... 22% 111M 1s 2024-03-04T18:06:46.102 INFO:teuthology.orchestra.run.smithi137.stderr: 20550K .......... .......... .......... .......... .......... 22% 116M 1s 2024-03-04T18:06:46.102 INFO:teuthology.orchestra.run.smithi137.stderr: 20600K .......... .......... .......... .......... .......... 22% 111M 1s 2024-03-04T18:06:46.102 INFO:teuthology.orchestra.run.smithi137.stderr: 20650K .......... .......... .......... .......... .......... 22% 101M 1s 2024-03-04T18:06:46.102 INFO:teuthology.orchestra.run.smithi137.stderr: 20700K .......... .......... .......... .......... .......... 22% 124M 1s 2024-03-04T18:06:46.102 INFO:teuthology.orchestra.run.smithi137.stderr: 20750K .......... .......... .......... .......... .......... 22% 110M 1s 2024-03-04T18:06:46.102 INFO:teuthology.orchestra.run.smithi137.stderr: 20800K .......... .......... .......... .......... .......... 22% 95.3M 1s 2024-03-04T18:06:46.102 INFO:teuthology.orchestra.run.smithi137.stderr: 20850K .......... .......... .......... .......... .......... 22% 88.9M 1s 2024-03-04T18:06:46.102 INFO:teuthology.orchestra.run.smithi137.stderr: 20900K .......... .......... .......... .......... .......... 22% 118M 1s 2024-03-04T18:06:46.102 INFO:teuthology.orchestra.run.smithi137.stderr: 20950K .......... .......... .......... .......... .......... 22% 113M 1s 2024-03-04T18:06:46.102 INFO:teuthology.orchestra.run.smithi137.stderr: 21000K .......... .......... .......... .......... .......... 22% 116M 1s 2024-03-04T18:06:46.102 INFO:teuthology.orchestra.run.smithi137.stderr: 21050K .......... .......... .......... .......... .......... 22% 98.9M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21100K .......... .......... .......... .......... .......... 22% 116M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21150K .......... .......... .......... .......... .......... 22% 104M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21200K .......... .......... .......... .......... .......... 22% 136M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21250K .......... .......... .......... .......... .......... 22% 91.1M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21300K .......... .......... .......... .......... .......... 22% 53.1M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21350K .......... .......... .......... .......... .......... 23% 113M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21400K .......... .......... .......... .......... .......... 23% 114M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21450K .......... .......... .......... .......... .......... 23% 106M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21500K .......... .......... .......... .......... .......... 23% 120M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21550K .......... .......... .......... .......... .......... 23% 74.7M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21600K .......... .......... .......... .......... .......... 23% 89.6M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21650K .......... .......... .......... .......... .......... 23% 34.9M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21700K .......... .......... .......... .......... .......... 23% 120M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21750K .......... .......... .......... .......... .......... 23% 63.7M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21800K .......... .......... .......... .......... .......... 23% 113M 1s 2024-03-04T18:06:46.103 INFO:teuthology.orchestra.run.smithi137.stderr: 21850K .......... .......... .......... .......... .......... 23% 108M 1s 2024-03-04T18:06:46.104 INFO:teuthology.orchestra.run.smithi137.stderr: 21900K .......... .......... .......... .......... .......... 23% 116M 1s 2024-03-04T18:06:46.104 INFO:teuthology.orchestra.run.smithi137.stderr: 21950K .......... .......... .......... .......... .......... 23% 105M 1s 2024-03-04T18:06:46.104 INFO:teuthology.orchestra.run.smithi137.stderr: 22000K .......... .......... .......... .......... .......... 23% 114M 1s 2024-03-04T18:06:46.104 INFO:teuthology.orchestra.run.smithi137.stderr: 22050K .......... .......... .......... .......... .......... 23% 88.5M 1s 2024-03-04T18:06:46.104 INFO:teuthology.orchestra.run.smithi137.stderr: 22100K .......... .......... .......... .......... .......... 23% 107M 1s 2024-03-04T18:06:46.104 INFO:teuthology.orchestra.run.smithi137.stderr: 22150K .......... .......... .......... .......... .......... 23% 126M 1s 2024-03-04T18:06:46.104 INFO:teuthology.orchestra.run.smithi137.stderr: 22200K .......... .......... .......... .......... .......... 23% 109M 1s 2024-03-04T18:06:46.104 INFO:teuthology.orchestra.run.smithi137.stderr: 22250K .......... .......... .......... .......... .......... 24% 99.1M 1s 2024-03-04T18:06:46.104 INFO:teuthology.orchestra.run.smithi137.stderr: 22300K .......... .......... .......... .......... .......... 24% 116M 1s 2024-03-04T18:06:46.104 INFO:teuthology.orchestra.run.smithi137.stderr: 22350K .......... .......... .......... .......... .......... 24% 114M 1s 2024-03-04T18:06:46.104 INFO:teuthology.orchestra.run.smithi137.stderr: 22400K .......... .......... .......... .......... .......... 24% 112M 1s 2024-03-04T18:06:46.104 INFO:teuthology.orchestra.run.smithi137.stderr: 22450K .......... .......... .......... .......... .......... 24% 87.1M 1s 2024-03-04T18:06:46.104 INFO:teuthology.orchestra.run.smithi137.stderr: 22500K .......... .......... .......... .......... .......... 24% 108M 1s 2024-03-04T18:06:46.105 INFO:teuthology.orchestra.run.smithi137.stderr: 22550K .......... .......... .......... .......... .......... 24% 106M 1s 2024-03-04T18:06:46.105 INFO:teuthology.orchestra.run.smithi137.stderr: 22600K .......... .......... .......... .......... .......... 24% 114M 1s 2024-03-04T18:06:46.105 INFO:teuthology.orchestra.run.smithi137.stderr: 22650K .......... .......... .......... .......... .......... 24% 109M 1s 2024-03-04T18:06:46.105 INFO:teuthology.orchestra.run.smithi137.stderr: 22700K .......... .......... .......... .......... .......... 24% 110M 1s 2024-03-04T18:06:46.105 INFO:teuthology.orchestra.run.smithi137.stderr: 22750K .......... .......... .......... .......... .......... 24% 106M 1s 2024-03-04T18:06:46.105 INFO:teuthology.orchestra.run.smithi137.stderr: 22800K .......... .......... .......... .......... .......... 24% 126M 1s 2024-03-04T18:06:46.105 INFO:teuthology.orchestra.run.smithi137.stderr: 22850K .......... .......... .......... .......... .......... 24% 88.8M 1s 2024-03-04T18:06:46.105 INFO:teuthology.orchestra.run.smithi137.stderr: 22900K .......... .......... .......... .......... .......... 24% 111M 1s 2024-03-04T18:06:46.105 INFO:teuthology.orchestra.run.smithi137.stderr: 22950K .......... .......... .......... .......... .......... 24% 118M 1s 2024-03-04T18:06:46.105 INFO:teuthology.orchestra.run.smithi137.stderr: 23000K .......... .......... .......... .......... .......... 24% 107M 1s 2024-03-04T18:06:46.105 INFO:teuthology.orchestra.run.smithi137.stderr: 23050K .......... .......... .......... .......... .......... 24% 112M 1s 2024-03-04T18:06:46.105 INFO:teuthology.orchestra.run.smithi137.stderr: 23100K .......... .......... .......... .......... .......... 24% 113M 1s 2024-03-04T18:06:46.106 INFO:teuthology.orchestra.run.smithi137.stderr: 23150K .......... .......... .......... .......... .......... 24% 107M 1s 2024-03-04T18:06:46.106 INFO:teuthology.orchestra.run.smithi137.stderr: 23200K .......... .......... .......... .......... .......... 25% 119M 1s 2024-03-04T18:06:46.106 INFO:teuthology.orchestra.run.smithi137.stderr: 23250K .......... .......... .......... .......... .......... 25% 88.8M 1s 2024-03-04T18:06:46.106 INFO:teuthology.orchestra.run.smithi137.stderr: 23300K .......... .......... .......... .......... .......... 25% 116M 1s 2024-03-04T18:06:46.106 INFO:teuthology.orchestra.run.smithi137.stderr: 23350K .......... .......... .......... .......... .......... 25% 101M 1s 2024-03-04T18:06:46.106 INFO:teuthology.orchestra.run.smithi137.stderr: 23400K .......... .......... .......... .......... .......... 25% 118M 1s 2024-03-04T18:06:46.106 INFO:teuthology.orchestra.run.smithi137.stderr: 23450K .......... .......... .......... .......... .......... 25% 105M 1s 2024-03-04T18:06:46.106 INFO:teuthology.orchestra.run.smithi137.stderr: 23500K .......... .......... .......... .......... .......... 25% 120M 1s 2024-03-04T18:06:46.106 INFO:teuthology.orchestra.run.smithi137.stderr: 23550K .......... .......... .......... .......... .......... 25% 105M 1s 2024-03-04T18:06:46.106 INFO:teuthology.orchestra.run.smithi137.stderr: 23600K .......... .......... .......... .......... .......... 25% 115M 1s 2024-03-04T18:06:46.106 INFO:teuthology.orchestra.run.smithi137.stderr: 23650K .......... .......... .......... .......... .......... 25% 91.5M 1s 2024-03-04T18:06:46.107 INFO:teuthology.orchestra.run.smithi137.stderr: 23700K .......... .......... .......... .......... .......... 25% 110M 1s 2024-03-04T18:06:46.107 INFO:teuthology.orchestra.run.smithi137.stderr: 23750K .......... .......... .......... .......... .......... 25% 119M 1s 2024-03-04T18:06:46.107 INFO:teuthology.orchestra.run.smithi137.stderr: 23800K .......... .......... .......... .......... .......... 25% 113M 1s 2024-03-04T18:06:46.107 INFO:teuthology.orchestra.run.smithi137.stderr: 23850K .......... .......... .......... .......... .......... 25% 101M 1s 2024-03-04T18:06:46.107 INFO:teuthology.orchestra.run.smithi137.stderr: 23900K .......... .......... .......... .......... .......... 25% 101M 1s 2024-03-04T18:06:46.107 INFO:teuthology.orchestra.run.smithi137.stderr: 23950K .......... .......... .......... .......... .......... 25% 120M 1s 2024-03-04T18:06:46.107 INFO:teuthology.orchestra.run.smithi137.stderr: 24000K .......... .......... .......... .......... .......... 25% 120M 1s 2024-03-04T18:06:46.107 INFO:teuthology.orchestra.run.smithi137.stderr: 24050K .......... .......... .......... .......... .......... 25% 86.9M 1s 2024-03-04T18:06:46.107 INFO:teuthology.orchestra.run.smithi137.stderr: 24100K .......... .......... .......... .......... .......... 25% 121M 1s 2024-03-04T18:06:46.107 INFO:teuthology.orchestra.run.smithi137.stderr: 24150K .......... .......... .......... .......... .......... 26% 109M 1s 2024-03-04T18:06:46.107 INFO:teuthology.orchestra.run.smithi137.stderr: 24200K .......... .......... .......... .......... .......... 26% 120M 1s 2024-03-04T18:06:46.107 INFO:teuthology.orchestra.run.smithi137.stderr: 24250K .......... .......... .......... .......... .......... 26% 104M 1s 2024-03-04T18:06:46.107 INFO:teuthology.orchestra.run.smithi137.stderr: 24300K .......... .......... .......... .......... .......... 26% 116M 1s 2024-03-04T18:06:46.108 INFO:teuthology.orchestra.run.smithi137.stderr: 24350K .......... .......... .......... .......... .......... 26% 106M 1s 2024-03-04T18:06:46.108 INFO:teuthology.orchestra.run.smithi137.stderr: 24400K .......... .......... .......... .......... .......... 26% 113M 1s 2024-03-04T18:06:46.108 INFO:teuthology.orchestra.run.smithi137.stderr: 24450K .......... .......... .......... .......... .......... 26% 88.0M 1s 2024-03-04T18:06:46.108 INFO:teuthology.orchestra.run.smithi137.stderr: 24500K .......... .......... .......... .......... .......... 26% 111M 1s 2024-03-04T18:06:46.108 INFO:teuthology.orchestra.run.smithi137.stderr: 24550K .......... .......... .......... .......... .......... 26% 117M 1s 2024-03-04T18:06:46.108 INFO:teuthology.orchestra.run.smithi137.stderr: 24600K .......... .......... .......... .......... .......... 26% 122M 1s 2024-03-04T18:06:46.108 INFO:teuthology.orchestra.run.smithi137.stderr: 24650K .......... .......... .......... .......... .......... 26% 102M 1s 2024-03-04T18:06:46.108 INFO:teuthology.orchestra.run.smithi137.stderr: 24700K .......... .......... .......... .......... .......... 26% 120M 1s 2024-03-04T18:06:46.108 INFO:teuthology.orchestra.run.smithi137.stderr: 24750K .......... .......... .......... .......... .......... 26% 99.7M 1s 2024-03-04T18:06:46.108 INFO:teuthology.orchestra.run.smithi137.stderr: 24800K .......... .......... .......... .......... .......... 26% 45.6M 1s 2024-03-04T18:06:46.108 INFO:teuthology.orchestra.run.smithi137.stderr: 24850K .......... .......... .......... .......... .......... 26% 36.3M 1s 2024-03-04T18:06:46.108 INFO:teuthology.orchestra.run.smithi137.stderr: 24900K .......... .......... .......... .......... .......... 26% 119M 1s 2024-03-04T18:06:46.109 INFO:teuthology.orchestra.run.smithi137.stderr: 24950K .......... .......... .......... .......... .......... 26% 112M 1s 2024-03-04T18:06:46.109 INFO:teuthology.orchestra.run.smithi137.stderr: 25000K .......... .......... .......... .......... .......... 26% 117M 1s 2024-03-04T18:06:46.109 INFO:teuthology.orchestra.run.smithi137.stderr: 25050K .......... .......... .......... .......... .......... 27% 101M 1s 2024-03-04T18:06:46.109 INFO:teuthology.orchestra.run.smithi137.stderr: 25100K .......... .......... .......... .......... .......... 27% 98.2M 1s 2024-03-04T18:06:46.109 INFO:teuthology.orchestra.run.smithi137.stderr: 25150K .......... .......... .......... .......... .......... 27% 140M 1s 2024-03-04T18:06:46.109 INFO:teuthology.orchestra.run.smithi137.stderr: 25200K .......... .......... .......... .......... .......... 27% 110M 1s 2024-03-04T18:06:46.109 INFO:teuthology.orchestra.run.smithi137.stderr: 25250K .......... .......... .......... .......... .......... 27% 89.6M 1s 2024-03-04T18:06:46.109 INFO:teuthology.orchestra.run.smithi137.stderr: 25300K .......... .......... .......... .......... .......... 27% 89.3M 1s 2024-03-04T18:06:46.109 INFO:teuthology.orchestra.run.smithi137.stderr: 25350K .......... .......... .......... .......... .......... 27% 111M 1s 2024-03-04T18:06:46.109 INFO:teuthology.orchestra.run.smithi137.stderr: 25400K .......... .......... .......... .......... .......... 27% 112M 1s 2024-03-04T18:06:46.109 INFO:teuthology.orchestra.run.smithi137.stderr: 25450K .......... .......... .......... .......... .......... 27% 107M 1s 2024-03-04T18:06:46.110 INFO:teuthology.orchestra.run.smithi137.stderr: 25500K .......... .......... .......... .......... .......... 27% 108M 1s 2024-03-04T18:06:46.110 INFO:teuthology.orchestra.run.smithi137.stderr: 25550K .......... .......... .......... .......... .......... 27% 118M 1s 2024-03-04T18:06:46.110 INFO:teuthology.orchestra.run.smithi137.stderr: 25600K .......... .......... .......... .......... .......... 27% 119M 1s 2024-03-04T18:06:46.110 INFO:teuthology.orchestra.run.smithi137.stderr: 25650K .......... .......... .......... .......... .......... 27% 85.4M 1s 2024-03-04T18:06:46.110 INFO:teuthology.orchestra.run.smithi137.stderr: 25700K .......... .......... .......... .......... .......... 27% 125M 1s 2024-03-04T18:06:46.110 INFO:teuthology.orchestra.run.smithi137.stderr: 25750K .......... .......... .......... .......... .......... 27% 109M 1s 2024-03-04T18:06:46.110 INFO:teuthology.orchestra.run.smithi137.stderr: 25800K .......... .......... .......... .......... .......... 27% 116M 1s 2024-03-04T18:06:46.110 INFO:teuthology.orchestra.run.smithi137.stderr: 25850K .......... .......... .......... .......... .......... 27% 106M 1s 2024-03-04T18:06:46.110 INFO:teuthology.orchestra.run.smithi137.stderr: 25900K .......... .......... .......... .......... .......... 27% 116M 1s 2024-03-04T18:06:46.110 INFO:teuthology.orchestra.run.smithi137.stderr: 25950K .......... .......... .......... .......... .......... 27% 47.2M 1s 2024-03-04T18:06:46.110 INFO:teuthology.orchestra.run.smithi137.stderr: 26000K .......... .......... .......... .......... .......... 28% 108M 1s 2024-03-04T18:06:46.111 INFO:teuthology.orchestra.run.smithi137.stderr: 26050K .......... .......... .......... .......... .......... 28% 99.2M 1s 2024-03-04T18:06:46.111 INFO:teuthology.orchestra.run.smithi137.stderr: 26100K .......... .......... .......... .......... .......... 28% 112M 1s 2024-03-04T18:06:46.111 INFO:teuthology.orchestra.run.smithi137.stderr: 26150K .......... .......... .......... .......... .......... 28% 117M 1s 2024-03-04T18:06:46.111 INFO:teuthology.orchestra.run.smithi137.stderr: 26200K .......... .......... .......... .......... .......... 28% 106M 1s 2024-03-04T18:06:46.111 INFO:teuthology.orchestra.run.smithi137.stderr: 26250K .......... .......... .......... .......... .......... 28% 106M 1s 2024-03-04T18:06:46.111 INFO:teuthology.orchestra.run.smithi137.stderr: 26300K .......... .......... .......... .......... .......... 28% 109M 1s 2024-03-04T18:06:46.111 INFO:teuthology.orchestra.run.smithi137.stderr: 26350K .......... .......... .......... .......... .......... 28% 107M 1s 2024-03-04T18:06:46.111 INFO:teuthology.orchestra.run.smithi137.stderr: 26400K .......... .......... .......... .......... .......... 28% 127M 1s 2024-03-04T18:06:46.111 INFO:teuthology.orchestra.run.smithi137.stderr: 26450K .......... .......... .......... .......... .......... 28% 87.7M 1s 2024-03-04T18:06:46.111 INFO:teuthology.orchestra.run.smithi137.stderr: 26500K .......... .......... .......... .......... .......... 28% 118M 1s 2024-03-04T18:06:46.112 INFO:teuthology.orchestra.run.smithi137.stderr: 26550K .......... .......... .......... .......... .......... 28% 111M 1s 2024-03-04T18:06:46.112 INFO:teuthology.orchestra.run.smithi137.stderr: 26600K .......... .......... .......... .......... .......... 28% 115M 1s 2024-03-04T18:06:46.112 INFO:teuthology.orchestra.run.smithi137.stderr: 26650K .......... .......... .......... .......... .......... 28% 100M 1s 2024-03-04T18:06:46.112 INFO:teuthology.orchestra.run.smithi137.stderr: 26700K .......... .......... .......... .......... .......... 28% 121M 1s 2024-03-04T18:06:46.112 INFO:teuthology.orchestra.run.smithi137.stderr: 26750K .......... .......... .......... .......... .......... 28% 120M 1s 2024-03-04T18:06:46.112 INFO:teuthology.orchestra.run.smithi137.stderr: 26800K .......... .......... .......... .......... .......... 28% 109M 1s 2024-03-04T18:06:46.112 INFO:teuthology.orchestra.run.smithi137.stderr: 26850K .......... .......... .......... .......... .......... 28% 86.8M 1s 2024-03-04T18:06:46.112 INFO:teuthology.orchestra.run.smithi137.stderr: 26900K .......... .......... .......... .......... .......... 29% 114M 1s 2024-03-04T18:06:46.112 INFO:teuthology.orchestra.run.smithi137.stderr: 26950K .......... .......... .......... .......... .......... 29% 109M 1s 2024-03-04T18:06:46.112 INFO:teuthology.orchestra.run.smithi137.stderr: 27000K .......... .......... .......... .......... .......... 29% 102M 1s 2024-03-04T18:06:46.112 INFO:teuthology.orchestra.run.smithi137.stderr: 27050K .......... .......... .......... .......... .......... 29% 101M 1s 2024-03-04T18:06:46.112 INFO:teuthology.orchestra.run.smithi137.stderr: 27100K .......... .......... .......... .......... .......... 29% 134M 1s 2024-03-04T18:06:46.113 INFO:teuthology.orchestra.run.smithi137.stderr: 27150K .......... .......... .......... .......... .......... 29% 119M 1s 2024-03-04T18:06:46.113 INFO:teuthology.orchestra.run.smithi137.stderr: 27200K .......... .......... .......... .......... .......... 29% 114M 1s 2024-03-04T18:06:46.113 INFO:teuthology.orchestra.run.smithi137.stderr: 27250K .......... .......... .......... .......... .......... 29% 86.7M 1s 2024-03-04T18:06:46.113 INFO:teuthology.orchestra.run.smithi137.stderr: 27300K .......... .......... .......... .......... .......... 29% 120M 1s 2024-03-04T18:06:46.113 INFO:teuthology.orchestra.run.smithi137.stderr: 27350K .......... .......... .......... .......... .......... 29% 109M 1s 2024-03-04T18:06:46.113 INFO:teuthology.orchestra.run.smithi137.stderr: 27400K .......... .......... .......... .......... .......... 29% 110M 1s 2024-03-04T18:06:46.113 INFO:teuthology.orchestra.run.smithi137.stderr: 27450K .......... .......... .......... .......... .......... 29% 101M 1s 2024-03-04T18:06:46.113 INFO:teuthology.orchestra.run.smithi137.stderr: 27500K .......... .......... .......... .......... .......... 29% 121M 1s 2024-03-04T18:06:46.113 INFO:teuthology.orchestra.run.smithi137.stderr: 27550K .......... .......... .......... .......... .......... 29% 109M 1s 2024-03-04T18:06:46.113 INFO:teuthology.orchestra.run.smithi137.stderr: 27600K .......... .......... .......... .......... .......... 29% 105M 1s 2024-03-04T18:06:46.113 INFO:teuthology.orchestra.run.smithi137.stderr: 27650K .......... .......... .......... .......... .......... 29% 94.4M 1s 2024-03-04T18:06:46.113 INFO:teuthology.orchestra.run.smithi137.stderr: 27700K .......... .......... .......... .......... .......... 29% 113M 1s 2024-03-04T18:06:46.114 INFO:teuthology.orchestra.run.smithi137.stderr: 27750K .......... .......... .......... .......... .......... 29% 111M 1s 2024-03-04T18:06:46.114 INFO:teuthology.orchestra.run.smithi137.stderr: 27800K .......... .......... .......... .......... .......... 29% 110M 1s 2024-03-04T18:06:46.114 INFO:teuthology.orchestra.run.smithi137.stderr: 27850K .......... .......... .......... .......... .......... 30% 85.1M 1s 2024-03-04T18:06:46.114 INFO:teuthology.orchestra.run.smithi137.stderr: 27900K .......... .......... .......... .......... .......... 30% 89.9M 1s 2024-03-04T18:06:46.114 INFO:teuthology.orchestra.run.smithi137.stderr: 27950K .......... .......... .......... .......... .......... 30% 101M 1s 2024-03-04T18:06:46.114 INFO:teuthology.orchestra.run.smithi137.stderr: 28000K .......... .......... .......... .......... .......... 30% 99.0M 1s 2024-03-04T18:06:46.114 INFO:teuthology.orchestra.run.smithi137.stderr: 28050K .......... .......... .......... .......... .......... 30% 86.5M 1s 2024-03-04T18:06:46.114 INFO:teuthology.orchestra.run.smithi137.stderr: 28100K .......... .......... .......... .......... .......... 30% 106M 1s 2024-03-04T18:06:46.114 INFO:teuthology.orchestra.run.smithi137.stderr: 28150K .......... .......... .......... .......... .......... 30% 112M 1s 2024-03-04T18:06:46.114 INFO:teuthology.orchestra.run.smithi137.stderr: 28200K .......... .......... .......... .......... .......... 30% 103M 1s 2024-03-04T18:06:46.114 INFO:teuthology.orchestra.run.smithi137.stderr: 28250K .......... .......... .......... .......... .......... 30% 102M 1s 2024-03-04T18:06:46.114 INFO:teuthology.orchestra.run.smithi137.stderr: 28300K .......... .......... .......... .......... .......... 30% 111M 1s 2024-03-04T18:06:46.115 INFO:teuthology.orchestra.run.smithi137.stderr: 28350K .......... .......... .......... .......... .......... 30% 116M 1s 2024-03-04T18:06:46.115 INFO:teuthology.orchestra.run.smithi137.stderr: 28400K .......... .......... .......... .......... .......... 30% 111M 1s 2024-03-04T18:06:46.115 INFO:teuthology.orchestra.run.smithi137.stderr: 28450K .......... .......... .......... .......... .......... 30% 88.8M 1s 2024-03-04T18:06:46.115 INFO:teuthology.orchestra.run.smithi137.stderr: 28500K .......... .......... .......... .......... .......... 30% 116M 1s 2024-03-04T18:06:46.115 INFO:teuthology.orchestra.run.smithi137.stderr: 28550K .......... .......... .......... .......... .......... 30% 106M 1s 2024-03-04T18:06:46.115 INFO:teuthology.orchestra.run.smithi137.stderr: 28600K .......... .......... .......... .......... .......... 30% 111M 1s 2024-03-04T18:06:46.115 INFO:teuthology.orchestra.run.smithi137.stderr: 28650K .......... .......... .......... .......... .......... 30% 105M 1s 2024-03-04T18:06:46.115 INFO:teuthology.orchestra.run.smithi137.stderr: 28700K .......... .......... .......... .......... .......... 30% 113M 1s 2024-03-04T18:06:46.115 INFO:teuthology.orchestra.run.smithi137.stderr: 28750K .......... .......... .......... .......... .......... 31% 116M 1s 2024-03-04T18:06:46.115 INFO:teuthology.orchestra.run.smithi137.stderr: 28800K .......... .......... .......... .......... .......... 31% 99.6M 1s 2024-03-04T18:06:46.115 INFO:teuthology.orchestra.run.smithi137.stderr: 28850K .......... .......... .......... .......... .......... 31% 95.5M 1s 2024-03-04T18:06:46.116 INFO:teuthology.orchestra.run.smithi137.stderr: 28900K .......... .......... .......... .......... .......... 31% 115M 1s 2024-03-04T18:06:46.116 INFO:teuthology.orchestra.run.smithi137.stderr: 28950K .......... .......... .......... .......... .......... 31% 115M 1s 2024-03-04T18:06:46.116 INFO:teuthology.orchestra.run.smithi137.stderr: 29000K .......... .......... .......... .......... .......... 31% 102M 1s 2024-03-04T18:06:46.116 INFO:teuthology.orchestra.run.smithi137.stderr: 29050K .......... .......... .......... .......... .......... 31% 110M 1s 2024-03-04T18:06:46.116 INFO:teuthology.orchestra.run.smithi137.stderr: 29100K .......... .......... .......... .......... .......... 31% 110M 1s 2024-03-04T18:06:46.116 INFO:teuthology.orchestra.run.smithi137.stderr: 29150K .......... .......... .......... .......... .......... 31% 122M 1s 2024-03-04T18:06:46.116 INFO:teuthology.orchestra.run.smithi137.stderr: 29200K .......... .......... .......... .......... .......... 31% 125M 1s 2024-03-04T18:06:46.116 INFO:teuthology.orchestra.run.smithi137.stderr: 29250K .......... .......... .......... .......... .......... 31% 95.8M 1s 2024-03-04T18:06:46.116 INFO:teuthology.orchestra.run.smithi137.stderr: 29300K .......... .......... .......... .......... .......... 31% 110M 1s 2024-03-04T18:06:46.116 INFO:teuthology.orchestra.run.smithi137.stderr: 29350K .......... .......... .......... .......... .......... 31% 118M 1s 2024-03-04T18:06:46.116 INFO:teuthology.orchestra.run.smithi137.stderr: 29400K .......... .......... .......... .......... .......... 31% 110M 1s 2024-03-04T18:06:46.116 INFO:teuthology.orchestra.run.smithi137.stderr: 29450K .......... .......... .......... .......... .......... 31% 103M 1s 2024-03-04T18:06:46.117 INFO:teuthology.orchestra.run.smithi137.stderr: 29500K .......... .......... .......... .......... .......... 31% 112M 1s 2024-03-04T18:06:46.117 INFO:teuthology.orchestra.run.smithi137.stderr: 29550K .......... .......... .......... .......... .......... 31% 114M 1s 2024-03-04T18:06:46.117 INFO:teuthology.orchestra.run.smithi137.stderr: 29600K .......... .......... .......... .......... .......... 31% 85.4M 1s 2024-03-04T18:06:46.117 INFO:teuthology.orchestra.run.smithi137.stderr: 29650K .......... .......... .......... .......... .......... 31% 90.9M 1s 2024-03-04T18:06:46.117 INFO:teuthology.orchestra.run.smithi137.stderr: 29700K .......... .......... .......... .......... .......... 32% 117M 1s 2024-03-04T18:06:46.117 INFO:teuthology.orchestra.run.smithi137.stderr: 29750K .......... .......... .......... .......... .......... 32% 108M 1s 2024-03-04T18:06:46.117 INFO:teuthology.orchestra.run.smithi137.stderr: 29800K .......... .......... .......... .......... .......... 32% 105M 1s 2024-03-04T18:06:46.117 INFO:teuthology.orchestra.run.smithi137.stderr: 29850K .......... .......... .......... .......... .......... 32% 105M 1s 2024-03-04T18:06:46.117 INFO:teuthology.orchestra.run.smithi137.stderr: 29900K .......... .......... .......... .......... .......... 32% 111M 1s 2024-03-04T18:06:46.117 INFO:teuthology.orchestra.run.smithi137.stderr: 29950K .......... .......... .......... .......... .......... 32% 105M 1s 2024-03-04T18:06:46.117 INFO:teuthology.orchestra.run.smithi137.stderr: 30000K .......... .......... .......... .......... .......... 32% 122M 1s 2024-03-04T18:06:46.117 INFO:teuthology.orchestra.run.smithi137.stderr: 30050K .......... .......... .......... .......... .......... 32% 77.4M 1s 2024-03-04T18:06:46.118 INFO:teuthology.orchestra.run.smithi137.stderr: 30100K .......... .......... .......... .......... .......... 32% 112M 1s 2024-03-04T18:06:46.118 INFO:teuthology.orchestra.run.smithi137.stderr: 30150K .......... .......... .......... .......... .......... 32% 120M 1s 2024-03-04T18:06:46.118 INFO:teuthology.orchestra.run.smithi137.stderr: 30200K .......... .......... .......... .......... .......... 32% 109M 1s 2024-03-04T18:06:46.118 INFO:teuthology.orchestra.run.smithi137.stderr: 30250K .......... .......... .......... .......... .......... 32% 103M 1s 2024-03-04T18:06:46.118 INFO:teuthology.orchestra.run.smithi137.stderr: 30300K .......... .......... .......... .......... .......... 32% 117M 1s 2024-03-04T18:06:46.118 INFO:teuthology.orchestra.run.smithi137.stderr: 30350K .......... .......... .......... .......... .......... 32% 113M 1s 2024-03-04T18:06:46.118 INFO:teuthology.orchestra.run.smithi137.stderr: 30400K .......... .......... .......... .......... .......... 32% 110M 1s 2024-03-04T18:06:46.118 INFO:teuthology.orchestra.run.smithi137.stderr: 30450K .......... .......... .......... .......... .......... 32% 89.7M 1s 2024-03-04T18:06:46.118 INFO:teuthology.orchestra.run.smithi137.stderr: 30500K .......... .......... .......... .......... .......... 32% 112M 1s 2024-03-04T18:06:46.118 INFO:teuthology.orchestra.run.smithi137.stderr: 30550K .......... .......... .......... .......... .......... 32% 114M 1s 2024-03-04T18:06:46.118 INFO:teuthology.orchestra.run.smithi137.stderr: 30600K .......... .......... .......... .......... .......... 32% 107M 1s 2024-03-04T18:06:46.118 INFO:teuthology.orchestra.run.smithi137.stderr: 30650K .......... .......... .......... .......... .......... 33% 106M 1s 2024-03-04T18:06:46.118 INFO:teuthology.orchestra.run.smithi137.stderr: 30700K .......... .......... .......... .......... .......... 33% 113M 1s 2024-03-04T18:06:46.118 INFO:teuthology.orchestra.run.smithi137.stderr: 30750K .......... .......... .......... .......... .......... 33% 114M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 30800K .......... .......... .......... .......... .......... 33% 112M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 30850K .......... .......... .......... .......... .......... 33% 87.8M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 30900K .......... .......... .......... .......... .......... 33% 88.6M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 30950K .......... .......... .......... .......... .......... 33% 73.2M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 31000K .......... .......... .......... .......... .......... 33% 115M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 31050K .......... .......... .......... .......... .......... 33% 106M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 31100K .......... .......... .......... .......... .......... 33% 115M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 31150K .......... .......... .......... .......... .......... 33% 112M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 31200K .......... .......... .......... .......... .......... 33% 119M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 31250K .......... .......... .......... .......... .......... 33% 62.7M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 31300K .......... .......... .......... .......... .......... 33% 57.6M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 31350K .......... .......... .......... .......... .......... 33% 84.0M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 31400K .......... .......... .......... .......... .......... 33% 86.2M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 31450K .......... .......... .......... .......... .......... 33% 94.1M 1s 2024-03-04T18:06:46.119 INFO:teuthology.orchestra.run.smithi137.stderr: 31500K .......... .......... .......... .......... .......... 33% 94.6M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 31550K .......... .......... .......... .......... .......... 34% 45.7M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 31600K .......... .......... .......... .......... .......... 34% 111M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 31650K .......... .......... .......... .......... .......... 34% 12.2M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 31700K .......... .......... .......... .......... .......... 34% 134M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 31750K .......... .......... .......... .......... .......... 34% 131M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 31800K .......... .......... .......... .......... .......... 34% 125M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 31850K .......... .......... .......... .......... .......... 34% 113M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 31900K .......... .......... .......... .......... .......... 34% 132M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 31950K .......... .......... .......... .......... .......... 34% 135M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 32000K .......... .......... .......... .......... .......... 34% 124M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 32050K .......... .......... .......... .......... .......... 34% 105M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 32100K .......... .......... .......... .......... .......... 34% 137M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 32150K .......... .......... .......... .......... .......... 34% 122M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 32200K .......... .......... .......... .......... .......... 34% 137M 1s 2024-03-04T18:06:46.120 INFO:teuthology.orchestra.run.smithi137.stderr: 32250K .......... .......... .......... .......... .......... 34% 118M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 32300K .......... .......... .......... .......... .......... 34% 129M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 32350K .......... .......... .......... .......... .......... 34% 134M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 32400K .......... .......... .......... .......... .......... 34% 118M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 32450K .......... .......... .......... .......... .......... 34% 102M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 32500K .......... .......... .......... .......... .......... 35% 124M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 32550K .......... .......... .......... .......... .......... 35% 121M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 32600K .......... .......... .......... .......... .......... 35% 127M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 32650K .......... .......... .......... .......... .......... 35% 111M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 32700K .......... .......... .......... .......... .......... 35% 127M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 32750K .......... .......... .......... .......... .......... 35% 126M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 32800K .......... .......... .......... .......... .......... 35% 121M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 32850K .......... .......... .......... .......... .......... 35% 92.3M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 32900K .......... .......... .......... .......... .......... 35% 123M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 32950K .......... .......... .......... .......... .......... 35% 128M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 33000K .......... .......... .......... .......... .......... 35% 136M 1s 2024-03-04T18:06:46.121 INFO:teuthology.orchestra.run.smithi137.stderr: 33050K .......... .......... .......... .......... .......... 35% 128M 1s 2024-03-04T18:06:46.122 INFO:teuthology.orchestra.run.smithi137.stderr: 33100K .......... .......... .......... .......... .......... 35% 140M 1s 2024-03-04T18:06:46.122 INFO:teuthology.orchestra.run.smithi137.stderr: 33150K .......... .......... .......... .......... .......... 35% 145M 1s 2024-03-04T18:06:46.122 INFO:teuthology.orchestra.run.smithi137.stderr: 33200K .......... .......... .......... .......... .......... 35% 134M 1s 2024-03-04T18:06:46.122 INFO:teuthology.orchestra.run.smithi137.stderr: 33250K .......... .......... .......... .......... .......... 35% 114M 1s 2024-03-04T18:06:46.122 INFO:teuthology.orchestra.run.smithi137.stderr: 33300K .......... .......... .......... .......... .......... 35% 151M 1s 2024-03-04T18:06:46.122 INFO:teuthology.orchestra.run.smithi137.stderr: 33350K .......... .......... .......... .......... .......... 35% 133M 1s 2024-03-04T18:06:46.122 INFO:teuthology.orchestra.run.smithi137.stderr: 33400K .......... .......... .......... .......... .......... 36% 132M 1s 2024-03-04T18:06:46.122 INFO:teuthology.orchestra.run.smithi137.stderr: 33450K .......... .......... .......... .......... .......... 36% 114M 1s 2024-03-04T18:06:46.122 INFO:teuthology.orchestra.run.smithi137.stderr: 33500K .......... .......... .......... .......... .......... 36% 142M 1s 2024-03-04T18:06:46.122 INFO:teuthology.orchestra.run.smithi137.stderr: 33550K .......... .......... .......... .......... .......... 36% 151M 1s 2024-03-04T18:06:46.122 INFO:teuthology.orchestra.run.smithi137.stderr: 33600K .......... .......... .......... .......... .......... 36% 143M 1s 2024-03-04T18:06:46.122 INFO:teuthology.orchestra.run.smithi137.stderr: 33650K .......... .......... .......... .......... .......... 36% 113M 1s 2024-03-04T18:06:46.122 INFO:teuthology.orchestra.run.smithi137.stderr: 33700K .......... .......... .......... .......... .......... 36% 138M 1s 2024-03-04T18:06:46.122 INFO:teuthology.orchestra.run.smithi137.stderr: 33750K .......... .......... .......... .......... .......... 36% 143M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 33800K .......... .......... .......... .......... .......... 36% 156M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 33850K .......... .......... .......... .......... .......... 36% 135M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 33900K .......... .......... .......... .......... .......... 36% 133M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 33950K .......... .......... .......... .......... .......... 36% 122M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 34000K .......... .......... .......... .......... .......... 36% 122M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 34050K .......... .......... .......... .......... .......... 36% 88.3M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 34100K .......... .......... .......... .......... .......... 36% 111M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 34150K .......... .......... .......... .......... .......... 36% 108M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 34200K .......... .......... .......... .......... .......... 36% 121M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 34250K .......... .......... .......... .......... .......... 36% 111M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 34300K .......... .......... .......... .......... .......... 36% 110M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 34350K .......... .......... .......... .......... .......... 37% 110M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 34400K .......... .......... .......... .......... .......... 37% 111M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 34450K .......... .......... .......... .......... .......... 37% 78.3M 1s 2024-03-04T18:06:46.123 INFO:teuthology.orchestra.run.smithi137.stderr: 34500K .......... .......... .......... .......... .......... 37% 118M 1s 2024-03-04T18:06:46.124 INFO:teuthology.orchestra.run.smithi137.stderr: 34550K .......... .......... .......... .......... .......... 37% 77.3M 1s 2024-03-04T18:06:46.124 INFO:teuthology.orchestra.run.smithi137.stderr: 34600K .......... .......... .......... .......... .......... 37% 52.0M 1s 2024-03-04T18:06:46.124 INFO:teuthology.orchestra.run.smithi137.stderr: 34650K .......... .......... .......... .......... .......... 37% 29.2M 1s 2024-03-04T18:06:46.124 INFO:teuthology.orchestra.run.smithi137.stderr: 34700K .......... .......... .......... .......... .......... 37% 101M 1s 2024-03-04T18:06:46.124 INFO:teuthology.orchestra.run.smithi137.stderr: 34750K .......... .......... .......... .......... .......... 37% 113M 1s 2024-03-04T18:06:46.124 INFO:teuthology.orchestra.run.smithi137.stderr: 34800K .......... .......... .......... .......... .......... 37% 102M 1s 2024-03-04T18:06:46.124 INFO:teuthology.orchestra.run.smithi137.stderr: 34850K .......... .......... .......... .......... .......... 37% 94.5M 1s 2024-03-04T18:06:46.124 INFO:teuthology.orchestra.run.smithi137.stderr: 34900K .......... .......... .......... .......... .......... 37% 103M 1s 2024-03-04T18:06:46.124 INFO:teuthology.orchestra.run.smithi137.stderr: 34950K .......... .......... .......... .......... .......... 37% 130M 1s 2024-03-04T18:06:46.124 INFO:teuthology.orchestra.run.smithi137.stderr: 35000K .......... .......... .......... .......... .......... 37% 112M 1s 2024-03-04T18:06:46.124 INFO:teuthology.orchestra.run.smithi137.stderr: 35050K .......... .......... .......... .......... .......... 37% 106M 1s 2024-03-04T18:06:46.124 INFO:teuthology.orchestra.run.smithi137.stderr: 35100K .......... .......... .......... .......... .......... 37% 120M 1s 2024-03-04T18:06:46.124 INFO:teuthology.orchestra.run.smithi137.stderr: 35150K .......... .......... .......... .......... .......... 37% 111M 1s 2024-03-04T18:06:46.124 INFO:teuthology.orchestra.run.smithi137.stderr: 35200K .......... .......... .......... .......... .......... 37% 115M 1s 2024-03-04T18:06:46.125 INFO:teuthology.orchestra.run.smithi137.stderr: 35250K .......... .......... .......... .......... .......... 37% 87.2M 1s 2024-03-04T18:06:46.125 INFO:teuthology.orchestra.run.smithi137.stderr: 35300K .......... .......... .......... .......... .......... 38% 116M 1s 2024-03-04T18:06:46.125 INFO:teuthology.orchestra.run.smithi137.stderr: 35350K .......... .......... .......... .......... .......... 38% 111M 1s 2024-03-04T18:06:46.125 INFO:teuthology.orchestra.run.smithi137.stderr: 35400K .......... .......... .......... .......... .......... 38% 119M 1s 2024-03-04T18:06:46.125 INFO:teuthology.orchestra.run.smithi137.stderr: 35450K .......... .......... .......... .......... .......... 38% 107M 1s 2024-03-04T18:06:46.125 INFO:teuthology.orchestra.run.smithi137.stderr: 35500K .......... .......... .......... .......... .......... 38% 108M 1s 2024-03-04T18:06:46.125 INFO:teuthology.orchestra.run.smithi137.stderr: 35550K .......... .......... .......... .......... .......... 38% 74.7M 1s 2024-03-04T18:06:46.125 INFO:teuthology.orchestra.run.smithi137.stderr: 35600K .......... .......... .......... .......... .......... 38% 105M 1s 2024-03-04T18:06:46.125 INFO:teuthology.orchestra.run.smithi137.stderr: 35650K .......... .......... .......... .......... .......... 38% 92.0M 1s 2024-03-04T18:06:46.125 INFO:teuthology.orchestra.run.smithi137.stderr: 35700K .......... .......... .......... .......... .......... 38% 109M 1s 2024-03-04T18:06:46.125 INFO:teuthology.orchestra.run.smithi137.stderr: 35750K .......... .......... .......... .......... .......... 38% 110M 1s 2024-03-04T18:06:46.125 INFO:teuthology.orchestra.run.smithi137.stderr: 35800K .......... .......... .......... .......... .......... 38% 118M 1s 2024-03-04T18:06:46.125 INFO:teuthology.orchestra.run.smithi137.stderr: 35850K .......... .......... .......... .......... .......... 38% 98.9M 1s 2024-03-04T18:06:46.125 INFO:teuthology.orchestra.run.smithi137.stderr: 35900K .......... .......... .......... .......... .......... 38% 123M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 35950K .......... .......... .......... .......... .......... 38% 112M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36000K .......... .......... .......... .......... .......... 38% 108M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36050K .......... .......... .......... .......... .......... 38% 87.3M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36100K .......... .......... .......... .......... .......... 38% 104M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36150K .......... .......... .......... .......... .......... 38% 129M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36200K .......... .......... .......... .......... .......... 39% 113M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36250K .......... .......... .......... .......... .......... 39% 116M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36300K .......... .......... .......... .......... .......... 39% 107M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36350K .......... .......... .......... .......... .......... 39% 122M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36400K .......... .......... .......... .......... .......... 39% 104M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36450K .......... .......... .......... .......... .......... 39% 89.7M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36500K .......... .......... .......... .......... .......... 39% 108M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36550K .......... .......... .......... .......... .......... 39% 110M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36600K .......... .......... .......... .......... .......... 39% 116M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36650K .......... .......... .......... .......... .......... 39% 106M 1s 2024-03-04T18:06:46.126 INFO:teuthology.orchestra.run.smithi137.stderr: 36700K .......... .......... .......... .......... .......... 39% 107M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 36750K .......... .......... .......... .......... .......... 39% 117M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 36800K .......... .......... .......... .......... .......... 39% 113M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 36850K .......... .......... .......... .......... .......... 39% 91.9M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 36900K .......... .......... .......... .......... .......... 39% 110M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 36950K .......... .......... .......... .......... .......... 39% 115M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 37000K .......... .......... .......... .......... .......... 39% 120M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 37050K .......... .......... .......... .......... .......... 39% 108M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 37100K .......... .......... .......... .......... .......... 39% 114M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 37150K .......... .......... .......... .......... .......... 40% 119M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 37200K .......... .......... .......... .......... .......... 40% 109M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 37250K .......... .......... .......... .......... .......... 40% 91.0M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 37300K .......... .......... .......... .......... .......... 40% 106M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 37350K .......... .......... .......... .......... .......... 40% 121M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 37400K .......... .......... .......... .......... .......... 40% 112M 1s 2024-03-04T18:06:46.127 INFO:teuthology.orchestra.run.smithi137.stderr: 37450K .......... .......... .......... .......... .......... 40% 95.6M 1s 2024-03-04T18:06:46.128 INFO:teuthology.orchestra.run.smithi137.stderr: 37500K .......... .......... .......... .......... .......... 40% 109M 1s 2024-03-04T18:06:46.128 INFO:teuthology.orchestra.run.smithi137.stderr: 37550K .......... .......... .......... .......... .......... 40% 118M 1s 2024-03-04T18:06:46.128 INFO:teuthology.orchestra.run.smithi137.stderr: 37600K .......... .......... .......... .......... .......... 40% 99.7M 1s 2024-03-04T18:06:46.128 INFO:teuthology.orchestra.run.smithi137.stderr: 37650K .......... .......... .......... .......... .......... 40% 86.5M 1s 2024-03-04T18:06:46.128 INFO:teuthology.orchestra.run.smithi137.stderr: 37700K .......... .......... .......... .......... .......... 40% 113M 1s 2024-03-04T18:06:46.128 INFO:teuthology.orchestra.run.smithi137.stderr: 37750K .......... .......... .......... .......... .......... 40% 107M 1s 2024-03-04T18:06:46.128 INFO:teuthology.orchestra.run.smithi137.stderr: 37800K .......... .......... .......... .......... .......... 40% 120M 1s 2024-03-04T18:06:46.128 INFO:teuthology.orchestra.run.smithi137.stderr: 37850K .......... .......... .......... .......... .......... 40% 103M 1s 2024-03-04T18:06:46.128 INFO:teuthology.orchestra.run.smithi137.stderr: 37900K .......... .......... .......... .......... .......... 40% 114M 1s 2024-03-04T18:06:46.128 INFO:teuthology.orchestra.run.smithi137.stderr: 37950K .......... .......... .......... .......... .......... 40% 109M 1s 2024-03-04T18:06:46.128 INFO:teuthology.orchestra.run.smithi137.stderr: 38000K .......... .......... .......... .......... .......... 40% 121M 1s 2024-03-04T18:06:46.128 INFO:teuthology.orchestra.run.smithi137.stderr: 38050K .......... .......... .......... .......... .......... 41% 87.4M 1s 2024-03-04T18:06:46.128 INFO:teuthology.orchestra.run.smithi137.stderr: 38100K .......... .......... .......... .......... .......... 41% 106M 1s 2024-03-04T18:06:46.128 INFO:teuthology.orchestra.run.smithi137.stderr: 38150K .......... .......... .......... .......... .......... 41% 120M 1s 2024-03-04T18:06:46.129 INFO:teuthology.orchestra.run.smithi137.stderr: 38200K .......... .......... .......... .......... .......... 41% 103M 1s 2024-03-04T18:06:46.129 INFO:teuthology.orchestra.run.smithi137.stderr: 38250K .......... .......... .......... .......... .......... 41% 112M 1s 2024-03-04T18:06:46.129 INFO:teuthology.orchestra.run.smithi137.stderr: 38300K .......... .......... .......... .......... .......... 41% 122M 1s 2024-03-04T18:06:46.129 INFO:teuthology.orchestra.run.smithi137.stderr: 38350K .......... .......... .......... .......... .......... 41% 109M 1s 2024-03-04T18:06:46.129 INFO:teuthology.orchestra.run.smithi137.stderr: 38400K .......... .......... .......... .......... .......... 41% 96.7M 1s 2024-03-04T18:06:46.129 INFO:teuthology.orchestra.run.smithi137.stderr: 38450K .......... .......... .......... .......... .......... 41% 93.3M 1s 2024-03-04T18:06:46.129 INFO:teuthology.orchestra.run.smithi137.stderr: 38500K .......... .......... .......... .......... .......... 41% 114M 1s 2024-03-04T18:06:46.129 INFO:teuthology.orchestra.run.smithi137.stderr: 38550K .......... .......... .......... .......... .......... 41% 104M 1s 2024-03-04T18:06:46.129 INFO:teuthology.orchestra.run.smithi137.stderr: 38600K .......... .......... .......... .......... .......... 41% 117M 1s 2024-03-04T18:06:46.129 INFO:teuthology.orchestra.run.smithi137.stderr: 38650K .......... .......... .......... .......... .......... 41% 96.2M 1s 2024-03-04T18:06:46.129 INFO:teuthology.orchestra.run.smithi137.stderr: 38700K .......... .......... .......... .......... .......... 41% 119M 1s 2024-03-04T18:06:46.129 INFO:teuthology.orchestra.run.smithi137.stderr: 38750K .......... .......... .......... .......... .......... 41% 117M 1s 2024-03-04T18:06:46.129 INFO:teuthology.orchestra.run.smithi137.stderr: 38800K .......... .......... .......... .......... .......... 41% 110M 1s 2024-03-04T18:06:46.129 INFO:teuthology.orchestra.run.smithi137.stderr: 38850K .......... .......... .......... .......... .......... 41% 89.7M 1s 2024-03-04T18:06:46.130 INFO:teuthology.orchestra.run.smithi137.stderr: 38900K .......... .......... .......... .......... .......... 41% 107M 1s 2024-03-04T18:06:46.130 INFO:teuthology.orchestra.run.smithi137.stderr: 38950K .......... .......... .......... .......... .......... 41% 122M 1s 2024-03-04T18:06:46.130 INFO:teuthology.orchestra.run.smithi137.stderr: 39000K .......... .......... .......... .......... .......... 42% 106M 1s 2024-03-04T18:06:46.130 INFO:teuthology.orchestra.run.smithi137.stderr: 39050K .......... .......... .......... .......... .......... 42% 103M 1s 2024-03-04T18:06:46.130 INFO:teuthology.orchestra.run.smithi137.stderr: 39100K .......... .......... .......... .......... .......... 42% 113M 1s 2024-03-04T18:06:46.130 INFO:teuthology.orchestra.run.smithi137.stderr: 39150K .......... .......... .......... .......... .......... 42% 112M 1s 2024-03-04T18:06:46.130 INFO:teuthology.orchestra.run.smithi137.stderr: 39200K .......... .......... .......... .......... .......... 42% 119M 1s 2024-03-04T18:06:46.130 INFO:teuthology.orchestra.run.smithi137.stderr: 39250K .......... .......... .......... .......... .......... 42% 85.7M 1s 2024-03-04T18:06:46.130 INFO:teuthology.orchestra.run.smithi137.stderr: 39300K .......... .......... .......... .......... .......... 42% 106M 1s 2024-03-04T18:06:46.130 INFO:teuthology.orchestra.run.smithi137.stderr: 39350K .......... .......... .......... .......... .......... 42% 111M 1s 2024-03-04T18:06:46.130 INFO:teuthology.orchestra.run.smithi137.stderr: 39400K .......... .......... .......... .......... .......... 42% 116M 1s 2024-03-04T18:06:46.130 INFO:teuthology.orchestra.run.smithi137.stderr: 39450K .......... .......... .......... .......... .......... 42% 95.3M 1s 2024-03-04T18:06:46.130 INFO:teuthology.orchestra.run.smithi137.stderr: 39500K .......... .......... .......... .......... .......... 42% 108M 1s 2024-03-04T18:06:46.130 INFO:teuthology.orchestra.run.smithi137.stderr: 39550K .......... .......... .......... .......... .......... 42% 111M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 39600K .......... .......... .......... .......... .......... 42% 106M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 39650K .......... .......... .......... .......... .......... 42% 95.6M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 39700K .......... .......... .......... .......... .......... 42% 108M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 39750K .......... .......... .......... .......... .......... 42% 114M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 39800K .......... .......... .......... .......... .......... 42% 111M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 39850K .......... .......... .......... .......... .......... 42% 110M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 39900K .......... .......... .......... .......... .......... 43% 117M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 39950K .......... .......... .......... .......... .......... 43% 99.6M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 40000K .......... .......... .......... .......... .......... 43% 126M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 40050K .......... .......... .......... .......... .......... 43% 78.1M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 40100K .......... .......... .......... .......... .......... 43% 134M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 40150K .......... .......... .......... .......... .......... 43% 119M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 40200K .......... .......... .......... .......... .......... 43% 111M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 40250K .......... .......... .......... .......... .......... 43% 106M 1s 2024-03-04T18:06:46.131 INFO:teuthology.orchestra.run.smithi137.stderr: 40300K .......... .......... .......... .......... .......... 43% 115M 1s 2024-03-04T18:06:46.132 INFO:teuthology.orchestra.run.smithi137.stderr: 40350K .......... .......... .......... .......... .......... 43% 110M 1s 2024-03-04T18:06:46.132 INFO:teuthology.orchestra.run.smithi137.stderr: 40400K .......... .......... .......... .......... .......... 43% 94.5M 1s 2024-03-04T18:06:46.132 INFO:teuthology.orchestra.run.smithi137.stderr: 40450K .......... .......... .......... .......... .......... 43% 100M 1s 2024-03-04T18:06:46.132 INFO:teuthology.orchestra.run.smithi137.stderr: 40500K .......... .......... .......... .......... .......... 43% 108M 1s 2024-03-04T18:06:46.132 INFO:teuthology.orchestra.run.smithi137.stderr: 40550K .......... .......... .......... .......... .......... 43% 107M 1s 2024-03-04T18:06:46.132 INFO:teuthology.orchestra.run.smithi137.stderr: 40600K .......... .......... .......... .......... .......... 43% 113M 1s 2024-03-04T18:06:46.132 INFO:teuthology.orchestra.run.smithi137.stderr: 40650K .......... .......... .......... .......... .......... 43% 103M 1s 2024-03-04T18:06:46.132 INFO:teuthology.orchestra.run.smithi137.stderr: 40700K .......... .......... .......... .......... .......... 43% 110M 1s 2024-03-04T18:06:46.132 INFO:teuthology.orchestra.run.smithi137.stderr: 40750K .......... .......... .......... .......... .......... 43% 115M 1s 2024-03-04T18:06:46.132 INFO:teuthology.orchestra.run.smithi137.stderr: 40800K .......... .......... .......... .......... .......... 43% 107M 1s 2024-03-04T18:06:46.132 INFO:teuthology.orchestra.run.smithi137.stderr: 40850K .......... .......... .......... .......... .......... 44% 48.5M 1s 2024-03-04T18:06:46.132 INFO:teuthology.orchestra.run.smithi137.stderr: 40900K .......... .......... .......... .......... .......... 44% 115M 1s 2024-03-04T18:06:46.132 INFO:teuthology.orchestra.run.smithi137.stderr: 40950K .......... .......... .......... .......... .......... 44% 116M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41000K .......... .......... .......... .......... .......... 44% 44.4M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41050K .......... .......... .......... .......... .......... 44% 110M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41100K .......... .......... .......... .......... .......... 44% 122M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41150K .......... .......... .......... .......... .......... 44% 118M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41200K .......... .......... .......... .......... .......... 44% 109M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41250K .......... .......... .......... .......... .......... 44% 77.8M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41300K .......... .......... .......... .......... .......... 44% 134M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41350K .......... .......... .......... .......... .......... 44% 47.3M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41400K .......... .......... .......... .......... .......... 44% 118M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41450K .......... .......... .......... .......... .......... 44% 102M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41500K .......... .......... .......... .......... .......... 44% 113M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41550K .......... .......... .......... .......... .......... 44% 113M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41600K .......... .......... .......... .......... .......... 44% 99.2M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41650K .......... .......... .......... .......... .......... 44% 93.4M 1s 2024-03-04T18:06:46.133 INFO:teuthology.orchestra.run.smithi137.stderr: 41700K .......... .......... .......... .......... .......... 44% 104M 1s 2024-03-04T18:06:46.134 INFO:teuthology.orchestra.run.smithi137.stderr: 41750K .......... .......... .......... .......... .......... 44% 94.7M 1s 2024-03-04T18:06:46.134 INFO:teuthology.orchestra.run.smithi137.stderr: 41800K .......... .......... .......... .......... .......... 45% 127M 1s 2024-03-04T18:06:46.134 INFO:teuthology.orchestra.run.smithi137.stderr: 41850K .......... .......... .......... .......... .......... 45% 119M 1s 2024-03-04T18:06:46.134 INFO:teuthology.orchestra.run.smithi137.stderr: 41900K .......... .......... .......... .......... .......... 45% 124M 1s 2024-03-04T18:06:46.134 INFO:teuthology.orchestra.run.smithi137.stderr: 41950K .......... .......... .......... .......... .......... 45% 118M 1s 2024-03-04T18:06:46.134 INFO:teuthology.orchestra.run.smithi137.stderr: 42000K .......... .......... .......... .......... .......... 45% 112M 1s 2024-03-04T18:06:46.134 INFO:teuthology.orchestra.run.smithi137.stderr: 42050K .......... .......... .......... .......... .......... 45% 87.3M 1s 2024-03-04T18:06:46.134 INFO:teuthology.orchestra.run.smithi137.stderr: 42100K .......... .......... .......... .......... .......... 45% 110M 1s 2024-03-04T18:06:46.134 INFO:teuthology.orchestra.run.smithi137.stderr: 42150K .......... .......... .......... .......... .......... 45% 121M 1s 2024-03-04T18:06:46.134 INFO:teuthology.orchestra.run.smithi137.stderr: 42200K .......... .......... .......... .......... .......... 45% 119M 1s 2024-03-04T18:06:46.134 INFO:teuthology.orchestra.run.smithi137.stderr: 42250K .......... .......... .......... .......... .......... 45% 96.4M 1s 2024-03-04T18:06:46.134 INFO:teuthology.orchestra.run.smithi137.stderr: 42300K .......... .......... .......... .......... .......... 45% 123M 1s 2024-03-04T18:06:46.134 INFO:teuthology.orchestra.run.smithi137.stderr: 42350K .......... .......... .......... .......... .......... 45% 108M 1s 2024-03-04T18:06:46.134 INFO:teuthology.orchestra.run.smithi137.stderr: 42400K .......... .......... .......... .......... .......... 45% 119M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 42450K .......... .......... .......... .......... .......... 45% 87.9M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 42500K .......... .......... .......... .......... .......... 45% 108M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 42550K .......... .......... .......... .......... .......... 45% 106M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 42600K .......... .......... .......... .......... .......... 45% 115M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 42650K .......... .......... .......... .......... .......... 45% 108M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 42700K .......... .......... .......... .......... .......... 46% 105M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 42750K .......... .......... .......... .......... .......... 46% 106M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 42800K .......... .......... .......... .......... .......... 46% 109M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 42850K .......... .......... .......... .......... .......... 46% 93.7M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 42900K .......... .......... .......... .......... .......... 46% 111M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 42950K .......... .......... .......... .......... .......... 46% 107M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 43000K .......... .......... .......... .......... .......... 46% 112M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 43050K .......... .......... .......... .......... .......... 46% 104M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 43100K .......... .......... .......... .......... .......... 46% 117M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 43150K .......... .......... .......... .......... .......... 46% 112M 1s 2024-03-04T18:06:46.135 INFO:teuthology.orchestra.run.smithi137.stderr: 43200K .......... .......... .......... .......... .......... 46% 107M 1s 2024-03-04T18:06:46.136 INFO:teuthology.orchestra.run.smithi137.stderr: 43250K .......... .......... .......... .......... .......... 46% 91.1M 1s 2024-03-04T18:06:46.136 INFO:teuthology.orchestra.run.smithi137.stderr: 43300K .......... .......... .......... .......... .......... 46% 108M 1s 2024-03-04T18:06:46.136 INFO:teuthology.orchestra.run.smithi137.stderr: 43350K .......... .......... .......... .......... .......... 46% 102M 1s 2024-03-04T18:06:46.136 INFO:teuthology.orchestra.run.smithi137.stderr: 43400K .......... .......... .......... .......... .......... 46% 128M 1s 2024-03-04T18:06:46.136 INFO:teuthology.orchestra.run.smithi137.stderr: 43450K .......... .......... .......... .......... .......... 46% 103M 1s 2024-03-04T18:06:46.136 INFO:teuthology.orchestra.run.smithi137.stderr: 43500K .......... .......... .......... .......... .......... 46% 108M 1s 2024-03-04T18:06:46.136 INFO:teuthology.orchestra.run.smithi137.stderr: 43550K .......... .......... .......... .......... .......... 46% 87.4M 1s 2024-03-04T18:06:46.136 INFO:teuthology.orchestra.run.smithi137.stderr: 43600K .......... .......... .......... .......... .......... 46% 117M 1s 2024-03-04T18:06:46.136 INFO:teuthology.orchestra.run.smithi137.stderr: 43650K .......... .......... .......... .......... .......... 47% 91.7M 1s 2024-03-04T18:06:46.136 INFO:teuthology.orchestra.run.smithi137.stderr: 43700K .......... .......... .......... .......... .......... 47% 106M 1s 2024-03-04T18:06:46.136 INFO:teuthology.orchestra.run.smithi137.stderr: 43750K .......... .......... .......... .......... .......... 47% 115M 1s 2024-03-04T18:06:46.136 INFO:teuthology.orchestra.run.smithi137.stderr: 43800K .......... .......... .......... .......... .......... 47% 111M 1s 2024-03-04T18:06:46.136 INFO:teuthology.orchestra.run.smithi137.stderr: 43850K .......... .......... .......... .......... .......... 47% 105M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 43900K .......... .......... .......... .......... .......... 47% 115M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 43950K .......... .......... .......... .......... .......... 47% 108M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 44000K .......... .......... .......... .......... .......... 47% 122M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 44050K .......... .......... .......... .......... .......... 47% 82.1M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 44100K .......... .......... .......... .......... .......... 47% 114M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 44150K .......... .......... .......... .......... .......... 47% 114M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 44200K .......... .......... .......... .......... .......... 47% 111M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 44250K .......... .......... .......... .......... .......... 47% 104M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 44300K .......... .......... .......... .......... .......... 47% 114M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 44350K .......... .......... .......... .......... .......... 47% 117M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 44400K .......... .......... .......... .......... .......... 47% 107M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 44450K .......... .......... .......... .......... .......... 47% 40.9M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 44500K .......... .......... .......... .......... .......... 47% 109M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 44550K .......... .......... .......... .......... .......... 48% 116M 1s 2024-03-04T18:06:46.137 INFO:teuthology.orchestra.run.smithi137.stderr: 44600K .......... .......... .......... .......... .......... 48% 37.9M 1s 2024-03-04T18:06:46.138 INFO:teuthology.orchestra.run.smithi137.stderr: 44650K .......... .......... .......... .......... .......... 48% 50.6M 1s 2024-03-04T18:06:46.138 INFO:teuthology.orchestra.run.smithi137.stderr: 44700K .......... .......... .......... .......... .......... 48% 115M 1s 2024-03-04T18:06:46.138 INFO:teuthology.orchestra.run.smithi137.stderr: 44750K .......... .......... .......... .......... .......... 48% 114M 1s 2024-03-04T18:06:46.138 INFO:teuthology.orchestra.run.smithi137.stderr: 44800K .......... .......... .......... .......... .......... 48% 114M 1s 2024-03-04T18:06:46.138 INFO:teuthology.orchestra.run.smithi137.stderr: 44850K .......... .......... .......... .......... .......... 48% 85.9M 1s 2024-03-04T18:06:46.138 INFO:teuthology.orchestra.run.smithi137.stderr: 44900K .......... .......... .......... .......... .......... 48% 114M 1s 2024-03-04T18:06:46.138 INFO:teuthology.orchestra.run.smithi137.stderr: 44950K .......... .......... .......... .......... .......... 48% 110M 1s 2024-03-04T18:06:46.138 INFO:teuthology.orchestra.run.smithi137.stderr: 45000K .......... .......... .......... .......... .......... 48% 121M 1s 2024-03-04T18:06:46.138 INFO:teuthology.orchestra.run.smithi137.stderr: 45050K .......... .......... .......... .......... .......... 48% 99.2M 1s 2024-03-04T18:06:46.138 INFO:teuthology.orchestra.run.smithi137.stderr: 45100K .......... .......... .......... .......... .......... 48% 118M 1s 2024-03-04T18:06:46.138 INFO:teuthology.orchestra.run.smithi137.stderr: 45150K .......... .......... .......... .......... .......... 48% 108M 1s 2024-03-04T18:06:46.138 INFO:teuthology.orchestra.run.smithi137.stderr: 45200K .......... .......... .......... .......... .......... 48% 117M 1s 2024-03-04T18:06:46.139 INFO:teuthology.orchestra.run.smithi137.stderr: 45250K .......... .......... .......... .......... .......... 48% 91.6M 1s 2024-03-04T18:06:46.139 INFO:teuthology.orchestra.run.smithi137.stderr: 45300K .......... .......... .......... .......... .......... 48% 111M 1s 2024-03-04T18:06:46.139 INFO:teuthology.orchestra.run.smithi137.stderr: 45350K .......... .......... .......... .......... .......... 48% 106M 1s 2024-03-04T18:06:46.139 INFO:teuthology.orchestra.run.smithi137.stderr: 45400K .......... .......... .......... .......... .......... 48% 109M 1s 2024-03-04T18:06:46.139 INFO:teuthology.orchestra.run.smithi137.stderr: 45450K .......... .......... .......... .......... .......... 48% 111M 1s 2024-03-04T18:06:46.139 INFO:teuthology.orchestra.run.smithi137.stderr: 45500K .......... .......... .......... .......... .......... 49% 98.6M 1s 2024-03-04T18:06:46.139 INFO:teuthology.orchestra.run.smithi137.stderr: 45550K .......... .......... .......... .......... .......... 49% 102M 1s 2024-03-04T18:06:46.139 INFO:teuthology.orchestra.run.smithi137.stderr: 45600K .......... .......... .......... .......... .......... 49% 113M 1s 2024-03-04T18:06:46.139 INFO:teuthology.orchestra.run.smithi137.stderr: 45650K .......... .......... .......... .......... .......... 49% 86.4M 1s 2024-03-04T18:06:46.139 INFO:teuthology.orchestra.run.smithi137.stderr: 45700K .......... .......... .......... .......... .......... 49% 100M 1s 2024-03-04T18:06:46.139 INFO:teuthology.orchestra.run.smithi137.stderr: 45750K .......... .......... .......... .......... .......... 49% 81.5M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 45800K .......... .......... .......... .......... .......... 49% 123M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 45850K .......... .......... .......... .......... .......... 49% 96.1M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 45900K .......... .......... .......... .......... .......... 49% 108M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 45950K .......... .......... .......... .......... .......... 49% 112M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 46000K .......... .......... .......... .......... .......... 49% 105M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 46050K .......... .......... .......... .......... .......... 49% 97.1M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 46100K .......... .......... .......... .......... .......... 49% 109M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 46150K .......... .......... .......... .......... .......... 49% 82.5M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 46200K .......... .......... .......... .......... .......... 49% 119M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 46250K .......... .......... .......... .......... .......... 49% 98.3M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 46300K .......... .......... .......... .......... .......... 49% 115M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 46350K .......... .......... .......... .......... .......... 49% 115M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 46400K .......... .......... .......... .......... .......... 49% 122M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 46450K .......... .......... .......... .......... .......... 50% 87.8M 1s 2024-03-04T18:06:46.140 INFO:teuthology.orchestra.run.smithi137.stderr: 46500K .......... .......... .......... .......... .......... 50% 112M 1s 2024-03-04T18:06:46.141 INFO:teuthology.orchestra.run.smithi137.stderr: 46550K .......... .......... .......... .......... .......... 50% 120M 1s 2024-03-04T18:06:46.141 INFO:teuthology.orchestra.run.smithi137.stderr: 46600K .......... .......... .......... .......... .......... 50% 110M 1s 2024-03-04T18:06:46.141 INFO:teuthology.orchestra.run.smithi137.stderr: 46650K .......... .......... .......... .......... .......... 50% 110M 1s 2024-03-04T18:06:46.141 INFO:teuthology.orchestra.run.smithi137.stderr: 46700K .......... .......... .......... .......... .......... 50% 108M 1s 2024-03-04T18:06:46.141 INFO:teuthology.orchestra.run.smithi137.stderr: 46750K .......... .......... .......... .......... .......... 50% 113M 1s 2024-03-04T18:06:46.141 INFO:teuthology.orchestra.run.smithi137.stderr: 46800K .......... .......... .......... .......... .......... 50% 114M 1s 2024-03-04T18:06:46.141 INFO:teuthology.orchestra.run.smithi137.stderr: 46850K .......... .......... .......... .......... .......... 50% 67.9M 1s 2024-03-04T18:06:46.141 INFO:teuthology.orchestra.run.smithi137.stderr: 46900K .......... .......... .......... .......... .......... 50% 164M 1s 2024-03-04T18:06:46.141 INFO:teuthology.orchestra.run.smithi137.stderr: 46950K .......... .......... .......... .......... .......... 50% 121M 0s 2024-03-04T18:06:46.141 INFO:teuthology.orchestra.run.smithi137.stderr: 47000K .......... .......... .......... .......... .......... 50% 117M 0s 2024-03-04T18:06:46.141 INFO:teuthology.orchestra.run.smithi137.stderr: 47050K .......... .......... .......... .......... .......... 50% 104M 0s 2024-03-04T18:06:46.141 INFO:teuthology.orchestra.run.smithi137.stderr: 47100K .......... .......... .......... .......... .......... 50% 122M 0s 2024-03-04T18:06:46.141 INFO:teuthology.orchestra.run.smithi137.stderr: 47150K .......... .......... .......... .......... .......... 50% 109M 0s 2024-03-04T18:06:46.142 INFO:teuthology.orchestra.run.smithi137.stderr: 47200K .......... .......... .......... .......... .......... 50% 118M 0s 2024-03-04T18:06:46.142 INFO:teuthology.orchestra.run.smithi137.stderr: 47250K .......... .......... .......... .......... .......... 50% 88.4M 0s 2024-03-04T18:06:46.142 INFO:teuthology.orchestra.run.smithi137.stderr: 47300K .......... .......... .......... .......... .......... 50% 115M 0s 2024-03-04T18:06:46.142 INFO:teuthology.orchestra.run.smithi137.stderr: 47350K .......... .......... .......... .......... .......... 51% 118M 0s 2024-03-04T18:06:46.142 INFO:teuthology.orchestra.run.smithi137.stderr: 47400K .......... .......... .......... .......... .......... 51% 110M 0s 2024-03-04T18:06:46.142 INFO:teuthology.orchestra.run.smithi137.stderr: 47450K .......... .......... .......... .......... .......... 51% 108M 0s 2024-03-04T18:06:46.142 INFO:teuthology.orchestra.run.smithi137.stderr: 47500K .......... .......... .......... .......... .......... 51% 104M 0s 2024-03-04T18:06:46.142 INFO:teuthology.orchestra.run.smithi137.stderr: 47550K .......... .......... .......... .......... .......... 51% 122M 0s 2024-03-04T18:06:46.143 INFO:teuthology.orchestra.run.smithi137.stderr: 47600K .......... .......... .......... .......... .......... 51% 105M 0s 2024-03-04T18:06:46.143 INFO:teuthology.orchestra.run.smithi137.stderr: 47650K .......... .......... .......... .......... .......... 51% 91.4M 0s 2024-03-04T18:06:46.143 INFO:teuthology.orchestra.run.smithi137.stderr: 47700K .......... .......... .......... .......... .......... 51% 86.3M 0s 2024-03-04T18:06:46.143 INFO:teuthology.orchestra.run.smithi137.stderr: 47750K .......... .......... .......... .......... .......... 51% 121M 0s 2024-03-04T18:06:46.143 INFO:teuthology.orchestra.run.smithi137.stderr: 47800K .......... .......... .......... .......... .......... 51% 109M 0s 2024-03-04T18:06:46.143 INFO:teuthology.orchestra.run.smithi137.stderr: 47850K .......... .......... .......... .......... .......... 51% 104M 0s 2024-03-04T18:06:46.143 INFO:teuthology.orchestra.run.smithi137.stderr: 47900K .......... .......... .......... .......... .......... 51% 122M 0s 2024-03-04T18:06:46.143 INFO:teuthology.orchestra.run.smithi137.stderr: 47950K .......... .......... .......... .......... .......... 51% 115M 0s 2024-03-04T18:06:46.143 INFO:teuthology.orchestra.run.smithi137.stderr: 48000K .......... .......... .......... .......... .......... 51% 119M 0s 2024-03-04T18:06:46.143 INFO:teuthology.orchestra.run.smithi137.stderr: 48050K .......... .......... .......... .......... .......... 51% 89.5M 0s 2024-03-04T18:06:46.143 INFO:teuthology.orchestra.run.smithi137.stderr: 48100K .......... .......... .......... .......... .......... 51% 108M 0s 2024-03-04T18:06:46.143 INFO:teuthology.orchestra.run.smithi137.stderr: 48150K .......... .......... .......... .......... .......... 51% 117M 0s 2024-03-04T18:06:46.143 INFO:teuthology.orchestra.run.smithi137.stderr: 48200K .......... .......... .......... .......... .......... 51% 92.0M 0s 2024-03-04T18:06:46.144 INFO:teuthology.orchestra.run.smithi137.stderr: 48250K .......... .......... .......... .......... .......... 51% 129M 0s 2024-03-04T18:06:46.144 INFO:teuthology.orchestra.run.smithi137.stderr: 48300K .......... .......... .......... .......... .......... 52% 108M 0s 2024-03-04T18:06:46.144 INFO:teuthology.orchestra.run.smithi137.stderr: 48350K .......... .......... .......... .......... .......... 52% 97.0M 0s 2024-03-04T18:06:46.144 INFO:teuthology.orchestra.run.smithi137.stderr: 48400K .......... .......... .......... .......... .......... 52% 126M 0s 2024-03-04T18:06:46.144 INFO:teuthology.orchestra.run.smithi137.stderr: 48450K .......... .......... .......... .......... .......... 52% 98.2M 0s 2024-03-04T18:06:46.144 INFO:teuthology.orchestra.run.smithi137.stderr: 48500K .......... .......... .......... .......... .......... 52% 104M 0s 2024-03-04T18:06:46.144 INFO:teuthology.orchestra.run.smithi137.stderr: 48550K .......... .......... .......... .......... .......... 52% 121M 0s 2024-03-04T18:06:46.144 INFO:teuthology.orchestra.run.smithi137.stderr: 48600K .......... .......... .......... .......... .......... 52% 114M 0s 2024-03-04T18:06:46.144 INFO:teuthology.orchestra.run.smithi137.stderr: 48650K .......... .......... .......... .......... .......... 52% 106M 0s 2024-03-04T18:06:46.144 INFO:teuthology.orchestra.run.smithi137.stderr: 48700K .......... .......... .......... .......... .......... 52% 116M 0s 2024-03-04T18:06:46.144 INFO:teuthology.orchestra.run.smithi137.stderr: 48750K .......... .......... .......... .......... .......... 52% 111M 0s 2024-03-04T18:06:46.144 INFO:teuthology.orchestra.run.smithi137.stderr: 48800K .......... .......... .......... .......... .......... 52% 105M 0s 2024-03-04T18:06:46.145 INFO:teuthology.orchestra.run.smithi137.stderr: 48850K .......... .......... .......... .......... .......... 52% 92.2M 0s 2024-03-04T18:06:46.145 INFO:teuthology.orchestra.run.smithi137.stderr: 48900K .......... .......... .......... .......... .......... 52% 113M 0s 2024-03-04T18:06:46.145 INFO:teuthology.orchestra.run.smithi137.stderr: 48950K .......... .......... .......... .......... .......... 52% 115M 0s 2024-03-04T18:06:46.145 INFO:teuthology.orchestra.run.smithi137.stderr: 49000K .......... .......... .......... .......... .......... 52% 117M 0s 2024-03-04T18:06:46.145 INFO:teuthology.orchestra.run.smithi137.stderr: 49050K .......... .......... .......... .......... .......... 52% 96.7M 0s 2024-03-04T18:06:46.145 INFO:teuthology.orchestra.run.smithi137.stderr: 49100K .......... .......... .......... .......... .......... 52% 120M 0s 2024-03-04T18:06:46.145 INFO:teuthology.orchestra.run.smithi137.stderr: 49150K .......... .......... .......... .......... .......... 52% 96.4M 0s 2024-03-04T18:06:46.145 INFO:teuthology.orchestra.run.smithi137.stderr: 49200K .......... .......... .......... .......... .......... 53% 137M 0s 2024-03-04T18:06:46.145 INFO:teuthology.orchestra.run.smithi137.stderr: 49250K .......... .......... .......... .......... .......... 53% 89.3M 0s 2024-03-04T18:06:46.145 INFO:teuthology.orchestra.run.smithi137.stderr: 49300K .......... .......... .......... .......... .......... 53% 112M 0s 2024-03-04T18:06:46.145 INFO:teuthology.orchestra.run.smithi137.stderr: 49350K .......... .......... .......... .......... .......... 53% 114M 0s 2024-03-04T18:06:46.145 INFO:teuthology.orchestra.run.smithi137.stderr: 49400K .......... .......... .......... .......... .......... 53% 113M 0s 2024-03-04T18:06:46.146 INFO:teuthology.orchestra.run.smithi137.stderr: 49450K .......... .......... .......... .......... .......... 53% 99.0M 0s 2024-03-04T18:06:46.146 INFO:teuthology.orchestra.run.smithi137.stderr: 49500K .......... .......... .......... .......... .......... 53% 115M 0s 2024-03-04T18:06:46.146 INFO:teuthology.orchestra.run.smithi137.stderr: 49550K .......... .......... .......... .......... .......... 53% 98.7M 0s 2024-03-04T18:06:46.146 INFO:teuthology.orchestra.run.smithi137.stderr: 49600K .......... .......... .......... .......... .......... 53% 98.9M 0s 2024-03-04T18:06:46.146 INFO:teuthology.orchestra.run.smithi137.stderr: 49650K .......... .......... .......... .......... .......... 53% 79.4M 0s 2024-03-04T18:06:46.146 INFO:teuthology.orchestra.run.smithi137.stderr: 49700K .......... .......... .......... .......... .......... 53% 125M 0s 2024-03-04T18:06:46.146 INFO:teuthology.orchestra.run.smithi137.stderr: 49750K .......... .......... .......... .......... .......... 53% 81.8M 0s 2024-03-04T18:06:46.146 INFO:teuthology.orchestra.run.smithi137.stderr: 49800K .......... .......... .......... .......... .......... 53% 65.2M 0s 2024-03-04T18:06:46.146 INFO:teuthology.orchestra.run.smithi137.stderr: 49850K .......... .......... .......... .......... .......... 53% 59.5M 0s 2024-03-04T18:06:46.146 INFO:teuthology.orchestra.run.smithi137.stderr: 49900K .......... .......... .......... .......... .......... 53% 169M 0s 2024-03-04T18:06:46.146 INFO:teuthology.orchestra.run.smithi137.stderr: 49950K .......... .......... .......... .......... .......... 53% 150M 0s 2024-03-04T18:06:46.147 INFO:teuthology.orchestra.run.smithi137.stderr: 50000K .......... .......... .......... .......... .......... 53% 158M 0s 2024-03-04T18:06:46.147 INFO:teuthology.orchestra.run.smithi137.stderr: 50050K .......... .......... .......... .......... .......... 53% 135M 0s 2024-03-04T18:06:46.147 INFO:teuthology.orchestra.run.smithi137.stderr: 50100K .......... .......... .......... .......... .......... 53% 114M 0s 2024-03-04T18:06:46.147 INFO:teuthology.orchestra.run.smithi137.stderr: 50150K .......... .......... .......... .......... .......... 54% 118M 0s 2024-03-04T18:06:46.147 INFO:teuthology.orchestra.run.smithi137.stderr: 50200K .......... .......... .......... .......... .......... 54% 106M 0s 2024-03-04T18:06:46.147 INFO:teuthology.orchestra.run.smithi137.stderr: 50250K .......... .......... .......... .......... .......... 54% 103M 0s 2024-03-04T18:06:46.147 INFO:teuthology.orchestra.run.smithi137.stderr: 50300K .......... .......... .......... .......... .......... 54% 128M 0s 2024-03-04T18:06:46.147 INFO:teuthology.orchestra.run.smithi137.stderr: 50350K .......... .......... .......... .......... .......... 54% 111M 0s 2024-03-04T18:06:46.147 INFO:teuthology.orchestra.run.smithi137.stderr: 50400K .......... .......... .......... .......... .......... 54% 75.2M 0s 2024-03-04T18:06:46.147 INFO:teuthology.orchestra.run.smithi137.stderr: 50450K .......... .......... .......... .......... .......... 54% 85.8M 0s 2024-03-04T18:06:46.147 INFO:teuthology.orchestra.run.smithi137.stderr: 50500K .......... .......... .......... .......... .......... 54% 109M 0s 2024-03-04T18:06:46.148 INFO:teuthology.orchestra.run.smithi137.stderr: 50550K .......... .......... .......... .......... .......... 54% 108M 0s 2024-03-04T18:06:46.148 INFO:teuthology.orchestra.run.smithi137.stderr: 50600K .......... .......... .......... .......... .......... 54% 105M 0s 2024-03-04T18:06:46.148 INFO:teuthology.orchestra.run.smithi137.stderr: 50650K .......... .......... .......... .......... .......... 54% 119M 0s 2024-03-04T18:06:46.148 INFO:teuthology.orchestra.run.smithi137.stderr: 50700K .......... .......... .......... .......... .......... 54% 36.0M 0s 2024-03-04T18:06:46.148 INFO:teuthology.orchestra.run.smithi137.stderr: 50750K .......... .......... .......... .......... .......... 54% 101M 0s 2024-03-04T18:06:46.148 INFO:teuthology.orchestra.run.smithi137.stderr: 50800K .......... .......... .......... .......... .......... 54% 122M 0s 2024-03-04T18:06:46.148 INFO:teuthology.orchestra.run.smithi137.stderr: 50850K .......... .......... .......... .......... .......... 54% 88.9M 0s 2024-03-04T18:06:46.148 INFO:teuthology.orchestra.run.smithi137.stderr: 50900K .......... .......... .......... .......... .......... 54% 103M 0s 2024-03-04T18:06:46.148 INFO:teuthology.orchestra.run.smithi137.stderr: 50950K .......... .......... .......... .......... .......... 54% 123M 0s 2024-03-04T18:06:46.148 INFO:teuthology.orchestra.run.smithi137.stderr: 51000K .......... .......... .......... .......... .......... 54% 119M 0s 2024-03-04T18:06:46.148 INFO:teuthology.orchestra.run.smithi137.stderr: 51050K .......... .......... .......... .......... .......... 55% 110M 0s 2024-03-04T18:06:46.148 INFO:teuthology.orchestra.run.smithi137.stderr: 51100K .......... .......... .......... .......... .......... 55% 109M 0s 2024-03-04T18:06:46.148 INFO:teuthology.orchestra.run.smithi137.stderr: 51150K .......... .......... .......... .......... .......... 55% 46.7M 0s 2024-03-04T18:06:46.149 INFO:teuthology.orchestra.run.smithi137.stderr: 51200K .......... .......... .......... .......... .......... 55% 113M 0s 2024-03-04T18:06:46.149 INFO:teuthology.orchestra.run.smithi137.stderr: 51250K .......... .......... .......... .......... .......... 55% 78.9M 0s 2024-03-04T18:06:46.149 INFO:teuthology.orchestra.run.smithi137.stderr: 51300K .......... .......... .......... .......... .......... 55% 126M 0s 2024-03-04T18:06:46.149 INFO:teuthology.orchestra.run.smithi137.stderr: 51350K .......... .......... .......... .......... .......... 55% 114M 0s 2024-03-04T18:06:46.149 INFO:teuthology.orchestra.run.smithi137.stderr: 51400K .......... .......... .......... .......... .......... 55% 116M 0s 2024-03-04T18:06:46.149 INFO:teuthology.orchestra.run.smithi137.stderr: 51450K .......... .......... .......... .......... .......... 55% 111M 0s 2024-03-04T18:06:46.149 INFO:teuthology.orchestra.run.smithi137.stderr: 51500K .......... .......... .......... .......... .......... 55% 107M 0s 2024-03-04T18:06:46.149 INFO:teuthology.orchestra.run.smithi137.stderr: 51550K .......... .......... .......... .......... .......... 55% 122M 0s 2024-03-04T18:06:46.149 INFO:teuthology.orchestra.run.smithi137.stderr: 51600K .......... .......... .......... .......... .......... 55% 108M 0s 2024-03-04T18:06:46.149 INFO:teuthology.orchestra.run.smithi137.stderr: 51650K .......... .......... .......... .......... .......... 55% 90.4M 0s 2024-03-04T18:06:46.149 INFO:teuthology.orchestra.run.smithi137.stderr: 51700K .......... .......... .......... .......... .......... 55% 105M 0s 2024-03-04T18:06:46.150 INFO:teuthology.orchestra.run.smithi137.stderr: 51750K .......... .......... .......... .......... .......... 55% 96.0M 0s 2024-03-04T18:06:46.150 INFO:teuthology.orchestra.run.smithi137.stderr: 51800K .......... .......... .......... .......... .......... 55% 120M 0s 2024-03-04T18:06:46.150 INFO:teuthology.orchestra.run.smithi137.stderr: 51850K .......... .......... .......... .......... .......... 55% 104M 0s 2024-03-04T18:06:46.150 INFO:teuthology.orchestra.run.smithi137.stderr: 51900K .......... .......... .......... .......... .......... 55% 115M 0s 2024-03-04T18:06:46.150 INFO:teuthology.orchestra.run.smithi137.stderr: 51950K .......... .......... .......... .......... .......... 55% 101M 0s 2024-03-04T18:06:46.150 INFO:teuthology.orchestra.run.smithi137.stderr: 52000K .......... .......... .......... .......... .......... 56% 116M 0s 2024-03-04T18:06:46.150 INFO:teuthology.orchestra.run.smithi137.stderr: 52050K .......... .......... .......... .......... .......... 56% 91.6M 0s 2024-03-04T18:06:46.150 INFO:teuthology.orchestra.run.smithi137.stderr: 52100K .......... .......... .......... .......... .......... 56% 109M 0s 2024-03-04T18:06:46.150 INFO:teuthology.orchestra.run.smithi137.stderr: 52150K .......... .......... .......... .......... .......... 56% 125M 0s 2024-03-04T18:06:46.150 INFO:teuthology.orchestra.run.smithi137.stderr: 52200K .......... .......... .......... .......... .......... 56% 109M 0s 2024-03-04T18:06:46.150 INFO:teuthology.orchestra.run.smithi137.stderr: 52250K .......... .......... .......... .......... .......... 56% 113M 0s 2024-03-04T18:06:46.150 INFO:teuthology.orchestra.run.smithi137.stderr: 52300K .......... .......... .......... .......... .......... 56% 113M 0s 2024-03-04T18:06:46.151 INFO:teuthology.orchestra.run.smithi137.stderr: 52350K .......... .......... .......... .......... .......... 56% 104M 0s 2024-03-04T18:06:46.151 INFO:teuthology.orchestra.run.smithi137.stderr: 52400K .......... .......... .......... .......... .......... 56% 125M 0s 2024-03-04T18:06:46.151 INFO:teuthology.orchestra.run.smithi137.stderr: 52450K .......... .......... .......... .......... .......... 56% 86.4M 0s 2024-03-04T18:06:46.151 INFO:teuthology.orchestra.run.smithi137.stderr: 52500K .......... .......... .......... .......... .......... 56% 117M 0s 2024-03-04T18:06:46.151 INFO:teuthology.orchestra.run.smithi137.stderr: 52550K .......... .......... .......... .......... .......... 56% 114M 0s 2024-03-04T18:06:46.151 INFO:teuthology.orchestra.run.smithi137.stderr: 52600K .......... .......... .......... .......... .......... 56% 115M 0s 2024-03-04T18:06:46.151 INFO:teuthology.orchestra.run.smithi137.stderr: 52650K .......... .......... .......... .......... .......... 56% 102M 0s 2024-03-04T18:06:46.151 INFO:teuthology.orchestra.run.smithi137.stderr: 52700K .......... .......... .......... .......... .......... 56% 116M 0s 2024-03-04T18:06:46.151 INFO:teuthology.orchestra.run.smithi137.stderr: 52750K .......... .......... .......... .......... .......... 56% 105M 0s 2024-03-04T18:06:46.151 INFO:teuthology.orchestra.run.smithi137.stderr: 52800K .......... .......... .......... .......... .......... 56% 115M 0s 2024-03-04T18:06:46.151 INFO:teuthology.orchestra.run.smithi137.stderr: 52850K .......... .......... .......... .......... .......... 56% 84.0M 0s 2024-03-04T18:06:46.151 INFO:teuthology.orchestra.run.smithi137.stderr: 52900K .......... .......... .......... .......... .......... 56% 112M 0s 2024-03-04T18:06:46.152 INFO:teuthology.orchestra.run.smithi137.stderr: 52950K .......... .......... .......... .......... .......... 57% 110M 0s 2024-03-04T18:06:46.152 INFO:teuthology.orchestra.run.smithi137.stderr: 53000K .......... .......... .......... .......... .......... 57% 105M 0s 2024-03-04T18:06:46.152 INFO:teuthology.orchestra.run.smithi137.stderr: 53050K .......... .......... .......... .......... .......... 57% 105M 0s 2024-03-04T18:06:46.152 INFO:teuthology.orchestra.run.smithi137.stderr: 53100K .......... .......... .......... .......... .......... 57% 116M 0s 2024-03-04T18:06:46.152 INFO:teuthology.orchestra.run.smithi137.stderr: 53150K .......... .......... .......... .......... .......... 57% 115M 0s 2024-03-04T18:06:46.152 INFO:teuthology.orchestra.run.smithi137.stderr: 53200K .......... .......... .......... .......... .......... 57% 117M 0s 2024-03-04T18:06:46.152 INFO:teuthology.orchestra.run.smithi137.stderr: 53250K .......... .......... .......... .......... .......... 57% 88.8M 0s 2024-03-04T18:06:46.152 INFO:teuthology.orchestra.run.smithi137.stderr: 53300K .......... .......... .......... .......... .......... 57% 104M 0s 2024-03-04T18:06:46.152 INFO:teuthology.orchestra.run.smithi137.stderr: 53350K .......... .......... .......... .......... .......... 57% 116M 0s 2024-03-04T18:06:46.152 INFO:teuthology.orchestra.run.smithi137.stderr: 53400K .......... .......... .......... .......... .......... 57% 123M 0s 2024-03-04T18:06:46.152 INFO:teuthology.orchestra.run.smithi137.stderr: 53450K .......... .......... .......... .......... .......... 57% 106M 0s 2024-03-04T18:06:46.153 INFO:teuthology.orchestra.run.smithi137.stderr: 53500K .......... .......... .......... .......... .......... 57% 112M 0s 2024-03-04T18:06:46.153 INFO:teuthology.orchestra.run.smithi137.stderr: 53550K .......... .......... .......... .......... .......... 57% 106M 0s 2024-03-04T18:06:46.153 INFO:teuthology.orchestra.run.smithi137.stderr: 53600K .......... .......... .......... .......... .......... 57% 109M 0s 2024-03-04T18:06:46.153 INFO:teuthology.orchestra.run.smithi137.stderr: 53650K .......... .......... .......... .......... .......... 57% 94.3M 0s 2024-03-04T18:06:46.153 INFO:teuthology.orchestra.run.smithi137.stderr: 53700K .......... .......... .......... .......... .......... 57% 109M 0s 2024-03-04T18:06:46.153 INFO:teuthology.orchestra.run.smithi137.stderr: 53750K .......... .......... .......... .......... .......... 57% 110M 0s 2024-03-04T18:06:46.153 INFO:teuthology.orchestra.run.smithi137.stderr: 53800K .......... .......... .......... .......... .......... 57% 109M 0s 2024-03-04T18:06:46.153 INFO:teuthology.orchestra.run.smithi137.stderr: 53850K .......... .......... .......... .......... .......... 58% 113M 0s 2024-03-04T18:06:46.153 INFO:teuthology.orchestra.run.smithi137.stderr: 53900K .......... .......... .......... .......... .......... 58% 104M 0s 2024-03-04T18:06:46.153 INFO:teuthology.orchestra.run.smithi137.stderr: 53950K .......... .......... .......... .......... .......... 58% 122M 0s 2024-03-04T18:06:46.153 INFO:teuthology.orchestra.run.smithi137.stderr: 54000K .......... .......... .......... .......... .......... 58% 115M 0s 2024-03-04T18:06:46.153 INFO:teuthology.orchestra.run.smithi137.stderr: 54050K .......... .......... .......... .......... .......... 58% 88.3M 0s 2024-03-04T18:06:46.154 INFO:teuthology.orchestra.run.smithi137.stderr: 54100K .......... .......... .......... .......... .......... 58% 109M 0s 2024-03-04T18:06:46.154 INFO:teuthology.orchestra.run.smithi137.stderr: 54150K .......... .......... .......... .......... .......... 58% 110M 0s 2024-03-04T18:06:46.154 INFO:teuthology.orchestra.run.smithi137.stderr: 54200K .......... .......... .......... .......... .......... 58% 116M 0s 2024-03-04T18:06:46.154 INFO:teuthology.orchestra.run.smithi137.stderr: 54250K .......... .......... .......... .......... .......... 58% 99.8M 0s 2024-03-04T18:06:46.154 INFO:teuthology.orchestra.run.smithi137.stderr: 54300K .......... .......... .......... .......... .......... 58% 103M 0s 2024-03-04T18:06:46.154 INFO:teuthology.orchestra.run.smithi137.stderr: 54350K .......... .......... .......... .......... .......... 58% 123M 0s 2024-03-04T18:06:46.154 INFO:teuthology.orchestra.run.smithi137.stderr: 54400K .......... .......... .......... .......... .......... 58% 34.0M 0s 2024-03-04T18:06:46.154 INFO:teuthology.orchestra.run.smithi137.stderr: 54450K .......... .......... .......... .......... .......... 58% 86.4M 0s 2024-03-04T18:06:46.154 INFO:teuthology.orchestra.run.smithi137.stderr: 54500K .......... .......... .......... .......... .......... 58% 117M 0s 2024-03-04T18:06:46.154 INFO:teuthology.orchestra.run.smithi137.stderr: 54550K .......... .......... .......... .......... .......... 58% 110M 0s 2024-03-04T18:06:46.154 INFO:teuthology.orchestra.run.smithi137.stderr: 54600K .......... .......... .......... .......... .......... 58% 112M 0s 2024-03-04T18:06:46.154 INFO:teuthology.orchestra.run.smithi137.stderr: 54650K .......... .......... .......... .......... .......... 58% 112M 0s 2024-03-04T18:06:46.154 INFO:teuthology.orchestra.run.smithi137.stderr: 54700K .......... .......... .......... .......... .......... 58% 113M 0s 2024-03-04T18:06:46.155 INFO:teuthology.orchestra.run.smithi137.stderr: 54750K .......... .......... .......... .......... .......... 58% 110M 0s 2024-03-04T18:06:46.155 INFO:teuthology.orchestra.run.smithi137.stderr: 54800K .......... .......... .......... .......... .......... 59% 115M 0s 2024-03-04T18:06:46.155 INFO:teuthology.orchestra.run.smithi137.stderr: 54850K .......... .......... .......... .......... .......... 59% 91.4M 0s 2024-03-04T18:06:46.155 INFO:teuthology.orchestra.run.smithi137.stderr: 54900K .......... .......... .......... .......... .......... 59% 111M 0s 2024-03-04T18:06:46.155 INFO:teuthology.orchestra.run.smithi137.stderr: 54950K .......... .......... .......... .......... .......... 59% 107M 0s 2024-03-04T18:06:46.155 INFO:teuthology.orchestra.run.smithi137.stderr: 55000K .......... .......... .......... .......... .......... 59% 100M 0s 2024-03-04T18:06:46.155 INFO:teuthology.orchestra.run.smithi137.stderr: 55050K .......... .......... .......... .......... .......... 59% 123M 0s 2024-03-04T18:06:46.155 INFO:teuthology.orchestra.run.smithi137.stderr: 55100K .......... .......... .......... .......... .......... 59% 88.5M 0s 2024-03-04T18:06:46.155 INFO:teuthology.orchestra.run.smithi137.stderr: 55150K .......... .......... .......... .......... .......... 59% 137M 0s 2024-03-04T18:06:46.155 INFO:teuthology.orchestra.run.smithi137.stderr: 55200K .......... .......... .......... .......... .......... 59% 114M 0s 2024-03-04T18:06:46.155 INFO:teuthology.orchestra.run.smithi137.stderr: 55250K .......... .......... .......... .......... .......... 59% 91.3M 0s 2024-03-04T18:06:46.155 INFO:teuthology.orchestra.run.smithi137.stderr: 55300K .......... .......... .......... .......... .......... 59% 113M 0s 2024-03-04T18:06:46.155 INFO:teuthology.orchestra.run.smithi137.stderr: 55350K .......... .......... .......... .......... .......... 59% 113M 0s 2024-03-04T18:06:46.156 INFO:teuthology.orchestra.run.smithi137.stderr: 55400K .......... .......... .......... .......... .......... 59% 110M 0s 2024-03-04T18:06:46.156 INFO:teuthology.orchestra.run.smithi137.stderr: 55450K .......... .......... .......... .......... .......... 59% 97.2M 0s 2024-03-04T18:06:46.156 INFO:teuthology.orchestra.run.smithi137.stderr: 55500K .......... .......... .......... .......... .......... 59% 117M 0s 2024-03-04T18:06:46.156 INFO:teuthology.orchestra.run.smithi137.stderr: 55550K .......... .......... .......... .......... .......... 59% 113M 0s 2024-03-04T18:06:46.156 INFO:teuthology.orchestra.run.smithi137.stderr: 55600K .......... .......... .......... .......... .......... 59% 92.4M 0s 2024-03-04T18:06:46.156 INFO:teuthology.orchestra.run.smithi137.stderr: 55650K .......... .......... .......... .......... .......... 59% 85.0M 0s 2024-03-04T18:06:46.156 INFO:teuthology.orchestra.run.smithi137.stderr: 55700K .......... .......... .......... .......... .......... 60% 96.9M 0s 2024-03-04T18:06:46.156 INFO:teuthology.orchestra.run.smithi137.stderr: 55750K .......... .......... .......... .......... .......... 60% 79.4M 0s 2024-03-04T18:06:46.156 INFO:teuthology.orchestra.run.smithi137.stderr: 55800K .......... .......... .......... .......... .......... 60% 66.8M 0s 2024-03-04T18:06:46.157 INFO:teuthology.orchestra.run.smithi137.stderr: 55850K .......... .......... .......... .......... .......... 60% 108M 0s 2024-03-04T18:06:46.157 INFO:teuthology.orchestra.run.smithi137.stderr: 55900K .......... .......... .......... .......... .......... 60% 114M 0s 2024-03-04T18:06:46.157 INFO:teuthology.orchestra.run.smithi137.stderr: 55950K .......... .......... .......... .......... .......... 60% 113M 0s 2024-03-04T18:06:46.157 INFO:teuthology.orchestra.run.smithi137.stderr: 56000K .......... .......... .......... .......... .......... 60% 103M 0s 2024-03-04T18:06:46.157 INFO:teuthology.orchestra.run.smithi137.stderr: 56050K .......... .......... .......... .......... .......... 60% 88.8M 0s 2024-03-04T18:06:46.157 INFO:teuthology.orchestra.run.smithi137.stderr: 56100K .......... .......... .......... .......... .......... 60% 117M 0s 2024-03-04T18:06:46.157 INFO:teuthology.orchestra.run.smithi137.stderr: 56150K .......... .......... .......... .......... .......... 60% 117M 0s 2024-03-04T18:06:46.157 INFO:teuthology.orchestra.run.smithi137.stderr: 56200K .......... .......... .......... .......... .......... 60% 113M 0s 2024-03-04T18:06:46.157 INFO:teuthology.orchestra.run.smithi137.stderr: 56250K .......... .......... .......... .......... .......... 60% 112M 0s 2024-03-04T18:06:46.157 INFO:teuthology.orchestra.run.smithi137.stderr: 56300K .......... .......... .......... .......... .......... 60% 107M 0s 2024-03-04T18:06:46.157 INFO:teuthology.orchestra.run.smithi137.stderr: 56350K .......... .......... .......... .......... .......... 60% 107M 0s 2024-03-04T18:06:46.157 INFO:teuthology.orchestra.run.smithi137.stderr: 56400K .......... .......... .......... .......... .......... 60% 118M 0s 2024-03-04T18:06:46.157 INFO:teuthology.orchestra.run.smithi137.stderr: 56450K .......... .......... .......... .......... .......... 60% 89.4M 0s 2024-03-04T18:06:46.158 INFO:teuthology.orchestra.run.smithi137.stderr: 56500K .......... .......... .......... .......... .......... 60% 117M 0s 2024-03-04T18:06:46.158 INFO:teuthology.orchestra.run.smithi137.stderr: 56550K .......... .......... .......... .......... .......... 60% 110M 0s 2024-03-04T18:06:46.158 INFO:teuthology.orchestra.run.smithi137.stderr: 56600K .......... .......... .......... .......... .......... 60% 114M 0s 2024-03-04T18:06:46.158 INFO:teuthology.orchestra.run.smithi137.stderr: 56650K .......... .......... .......... .......... .......... 61% 106M 0s 2024-03-04T18:06:46.158 INFO:teuthology.orchestra.run.smithi137.stderr: 56700K .......... .......... .......... .......... .......... 61% 119M 0s 2024-03-04T18:06:46.158 INFO:teuthology.orchestra.run.smithi137.stderr: 56750K .......... .......... .......... .......... .......... 61% 103M 0s 2024-03-04T18:06:46.158 INFO:teuthology.orchestra.run.smithi137.stderr: 56800K .......... .......... .......... .......... .......... 61% 106M 0s 2024-03-04T18:06:46.158 INFO:teuthology.orchestra.run.smithi137.stderr: 56850K .......... .......... .......... .......... .......... 61% 88.4M 0s 2024-03-04T18:06:46.158 INFO:teuthology.orchestra.run.smithi137.stderr: 56900K .......... .......... .......... .......... .......... 61% 109M 0s 2024-03-04T18:06:46.158 INFO:teuthology.orchestra.run.smithi137.stderr: 56950K .......... .......... .......... .......... .......... 61% 123M 0s 2024-03-04T18:06:46.158 INFO:teuthology.orchestra.run.smithi137.stderr: 57000K .......... .......... .......... .......... .......... 61% 113M 0s 2024-03-04T18:06:46.158 INFO:teuthology.orchestra.run.smithi137.stderr: 57050K .......... .......... .......... .......... .......... 61% 109M 0s 2024-03-04T18:06:46.158 INFO:teuthology.orchestra.run.smithi137.stderr: 57100K .......... .......... .......... .......... .......... 61% 109M 0s 2024-03-04T18:06:46.159 INFO:teuthology.orchestra.run.smithi137.stderr: 57150K .......... .......... .......... .......... .......... 61% 120M 0s 2024-03-04T18:06:46.159 INFO:teuthology.orchestra.run.smithi137.stderr: 57200K .......... .......... .......... .......... .......... 61% 111M 0s 2024-03-04T18:06:46.159 INFO:teuthology.orchestra.run.smithi137.stderr: 57250K .......... .......... .......... .......... .......... 61% 87.8M 0s 2024-03-04T18:06:46.159 INFO:teuthology.orchestra.run.smithi137.stderr: 57300K .......... .......... .......... .......... .......... 61% 119M 0s 2024-03-04T18:06:46.159 INFO:teuthology.orchestra.run.smithi137.stderr: 57350K .......... .......... .......... .......... .......... 61% 106M 0s 2024-03-04T18:06:46.159 INFO:teuthology.orchestra.run.smithi137.stderr: 57400K .......... .......... .......... .......... .......... 61% 121M 0s 2024-03-04T18:06:46.159 INFO:teuthology.orchestra.run.smithi137.stderr: 57450K .......... .......... .......... .......... .......... 61% 96.1M 0s 2024-03-04T18:06:46.159 INFO:teuthology.orchestra.run.smithi137.stderr: 57500K .......... .......... .......... .......... .......... 61% 121M 0s 2024-03-04T18:06:46.159 INFO:teuthology.orchestra.run.smithi137.stderr: 57550K .......... .......... .......... .......... .......... 62% 110M 0s 2024-03-04T18:06:46.159 INFO:teuthology.orchestra.run.smithi137.stderr: 57600K .......... .......... .......... .......... .......... 62% 120M 0s 2024-03-04T18:06:46.159 INFO:teuthology.orchestra.run.smithi137.stderr: 57650K .......... .......... .......... .......... .......... 62% 87.0M 0s 2024-03-04T18:06:46.160 INFO:teuthology.orchestra.run.smithi137.stderr: 57700K .......... .......... .......... .......... .......... 62% 111M 0s 2024-03-04T18:06:46.160 INFO:teuthology.orchestra.run.smithi137.stderr: 57750K .......... .......... .......... .......... .......... 62% 116M 0s 2024-03-04T18:06:46.160 INFO:teuthology.orchestra.run.smithi137.stderr: 57800K .......... .......... .......... .......... .......... 62% 115M 0s 2024-03-04T18:06:46.160 INFO:teuthology.orchestra.run.smithi137.stderr: 57850K .......... .......... .......... .......... .......... 62% 112M 0s 2024-03-04T18:06:46.160 INFO:teuthology.orchestra.run.smithi137.stderr: 57900K .......... .......... .......... .......... .......... 62% 111M 0s 2024-03-04T18:06:46.160 INFO:teuthology.orchestra.run.smithi137.stderr: 57950K .......... .......... .......... .......... .......... 62% 111M 0s 2024-03-04T18:06:46.160 INFO:teuthology.orchestra.run.smithi137.stderr: 58000K .......... .......... .......... .......... .......... 62% 102M 0s 2024-03-04T18:06:46.160 INFO:teuthology.orchestra.run.smithi137.stderr: 58050K .......... .......... .......... .......... .......... 62% 93.2M 0s 2024-03-04T18:06:46.160 INFO:teuthology.orchestra.run.smithi137.stderr: 58100K .......... .......... .......... .......... .......... 62% 121M 0s 2024-03-04T18:06:46.160 INFO:teuthology.orchestra.run.smithi137.stderr: 58150K .......... .......... .......... .......... .......... 62% 115M 0s 2024-03-04T18:06:46.160 INFO:teuthology.orchestra.run.smithi137.stderr: 58200K .......... .......... .......... .......... .......... 62% 113M 0s 2024-03-04T18:06:46.160 INFO:teuthology.orchestra.run.smithi137.stderr: 58250K .......... .......... .......... .......... .......... 62% 108M 0s 2024-03-04T18:06:46.160 INFO:teuthology.orchestra.run.smithi137.stderr: 58300K .......... .......... .......... .......... .......... 62% 117M 0s 2024-03-04T18:06:46.161 INFO:teuthology.orchestra.run.smithi137.stderr: 58350K .......... .......... .......... .......... .......... 62% 110M 0s 2024-03-04T18:06:46.161 INFO:teuthology.orchestra.run.smithi137.stderr: 58400K .......... .......... .......... .......... .......... 62% 113M 0s 2024-03-04T18:06:46.161 INFO:teuthology.orchestra.run.smithi137.stderr: 58450K .......... .......... .......... .......... .......... 62% 85.1M 0s 2024-03-04T18:06:46.161 INFO:teuthology.orchestra.run.smithi137.stderr: 58500K .......... .......... .......... .......... .......... 63% 123M 0s 2024-03-04T18:06:46.161 INFO:teuthology.orchestra.run.smithi137.stderr: 58550K .......... .......... .......... .......... .......... 63% 116M 0s 2024-03-04T18:06:46.161 INFO:teuthology.orchestra.run.smithi137.stderr: 58600K .......... .......... .......... .......... .......... 63% 108M 0s 2024-03-04T18:06:46.161 INFO:teuthology.orchestra.run.smithi137.stderr: 58650K .......... .......... .......... .......... .......... 63% 112M 0s 2024-03-04T18:06:46.161 INFO:teuthology.orchestra.run.smithi137.stderr: 58700K .......... .......... .......... .......... .......... 63% 92.3M 0s 2024-03-04T18:06:46.161 INFO:teuthology.orchestra.run.smithi137.stderr: 58750K .......... .......... .......... .......... .......... 63% 108M 0s 2024-03-04T18:06:46.161 INFO:teuthology.orchestra.run.smithi137.stderr: 58800K .......... .......... .......... .......... .......... 63% 112M 0s 2024-03-04T18:06:46.161 INFO:teuthology.orchestra.run.smithi137.stderr: 58850K .......... .......... .......... .......... .......... 63% 87.6M 0s 2024-03-04T18:06:46.161 INFO:teuthology.orchestra.run.smithi137.stderr: 58900K .......... .......... .......... .......... .......... 63% 111M 0s 2024-03-04T18:06:46.161 INFO:teuthology.orchestra.run.smithi137.stderr: 58950K .......... .......... .......... .......... .......... 63% 119M 0s 2024-03-04T18:06:46.162 INFO:teuthology.orchestra.run.smithi137.stderr: 59000K .......... .......... .......... .......... .......... 63% 117M 0s 2024-03-04T18:06:46.162 INFO:teuthology.orchestra.run.smithi137.stderr: 59050K .......... .......... .......... .......... .......... 63% 102M 0s 2024-03-04T18:06:46.162 INFO:teuthology.orchestra.run.smithi137.stderr: 59100K .......... .......... .......... .......... .......... 63% 116M 0s 2024-03-04T18:06:46.162 INFO:teuthology.orchestra.run.smithi137.stderr: 59150K .......... .......... .......... .......... .......... 63% 106M 0s 2024-03-04T18:06:46.162 INFO:teuthology.orchestra.run.smithi137.stderr: 59200K .......... .......... .......... .......... .......... 63% 117M 0s 2024-03-04T18:06:46.162 INFO:teuthology.orchestra.run.smithi137.stderr: 59250K .......... .......... .......... .......... .......... 63% 92.0M 0s 2024-03-04T18:06:46.162 INFO:teuthology.orchestra.run.smithi137.stderr: 59300K .......... .......... .......... .......... .......... 63% 108M 0s 2024-03-04T18:06:46.162 INFO:teuthology.orchestra.run.smithi137.stderr: 59350K .......... .......... .......... .......... .......... 63% 110M 0s 2024-03-04T18:06:46.162 INFO:teuthology.orchestra.run.smithi137.stderr: 59400K .......... .......... .......... .......... .......... 63% 112M 0s 2024-03-04T18:06:46.162 INFO:teuthology.orchestra.run.smithi137.stderr: 59450K .......... .......... .......... .......... .......... 64% 104M 0s 2024-03-04T18:06:46.162 INFO:teuthology.orchestra.run.smithi137.stderr: 59500K .......... .......... .......... .......... .......... 64% 118M 0s 2024-03-04T18:06:46.163 INFO:teuthology.orchestra.run.smithi137.stderr: 59550K .......... .......... .......... .......... .......... 64% 118M 0s 2024-03-04T18:06:46.163 INFO:teuthology.orchestra.run.smithi137.stderr: 59600K .......... .......... .......... .......... .......... 64% 110M 0s 2024-03-04T18:06:46.163 INFO:teuthology.orchestra.run.smithi137.stderr: 59650K .......... .......... .......... .......... .......... 64% 89.2M 0s 2024-03-04T18:06:46.163 INFO:teuthology.orchestra.run.smithi137.stderr: 59700K .......... .......... .......... .......... .......... 64% 111M 0s 2024-03-04T18:06:46.163 INFO:teuthology.orchestra.run.smithi137.stderr: 59750K .......... .......... .......... .......... .......... 64% 118M 0s 2024-03-04T18:06:46.163 INFO:teuthology.orchestra.run.smithi137.stderr: 59800K .......... .......... .......... .......... .......... 64% 114M 0s 2024-03-04T18:06:46.163 INFO:teuthology.orchestra.run.smithi137.stderr: 59850K .......... .......... .......... .......... .......... 64% 105M 0s 2024-03-04T18:06:46.163 INFO:teuthology.orchestra.run.smithi137.stderr: 59900K .......... .......... .......... .......... .......... 64% 108M 0s 2024-03-04T18:06:46.163 INFO:teuthology.orchestra.run.smithi137.stderr: 59950K .......... .......... .......... .......... .......... 64% 117M 0s 2024-03-04T18:06:46.163 INFO:teuthology.orchestra.run.smithi137.stderr: 60000K .......... .......... .......... .......... .......... 64% 107M 0s 2024-03-04T18:06:46.163 INFO:teuthology.orchestra.run.smithi137.stderr: 60050K .......... .......... .......... .......... .......... 64% 82.8M 0s 2024-03-04T18:06:46.163 INFO:teuthology.orchestra.run.smithi137.stderr: 60100K .......... .......... .......... .......... .......... 64% 107M 0s 2024-03-04T18:06:46.163 INFO:teuthology.orchestra.run.smithi137.stderr: 60150K .......... .......... .......... .......... .......... 64% 110M 0s 2024-03-04T18:06:46.164 INFO:teuthology.orchestra.run.smithi137.stderr: 60200K .......... .......... .......... .......... .......... 64% 107M 0s 2024-03-04T18:06:46.164 INFO:teuthology.orchestra.run.smithi137.stderr: 60250K .......... .......... .......... .......... .......... 64% 85.0M 0s 2024-03-04T18:06:46.164 INFO:teuthology.orchestra.run.smithi137.stderr: 60300K .......... .......... .......... .......... .......... 64% 107M 0s 2024-03-04T18:06:46.164 INFO:teuthology.orchestra.run.smithi137.stderr: 60350K .......... .......... .......... .......... .......... 65% 118M 0s 2024-03-04T18:06:46.164 INFO:teuthology.orchestra.run.smithi137.stderr: 60400K .......... .......... .......... .......... .......... 65% 114M 0s 2024-03-04T18:06:46.164 INFO:teuthology.orchestra.run.smithi137.stderr: 60450K .......... .......... .......... .......... .......... 65% 84.3M 0s 2024-03-04T18:06:46.164 INFO:teuthology.orchestra.run.smithi137.stderr: 60500K .......... .......... .......... .......... .......... 65% 112M 0s 2024-03-04T18:06:46.164 INFO:teuthology.orchestra.run.smithi137.stderr: 60550K .......... .......... .......... .......... .......... 65% 111M 0s 2024-03-04T18:06:46.164 INFO:teuthology.orchestra.run.smithi137.stderr: 60600K .......... .......... .......... .......... .......... 65% 89.3M 0s 2024-03-04T18:06:46.164 INFO:teuthology.orchestra.run.smithi137.stderr: 60650K .......... .......... .......... .......... .......... 65% 107M 0s 2024-03-04T18:06:46.164 INFO:teuthology.orchestra.run.smithi137.stderr: 60700K .......... .......... .......... .......... .......... 65% 113M 0s 2024-03-04T18:06:46.164 INFO:teuthology.orchestra.run.smithi137.stderr: 60750K .......... .......... .......... .......... .......... 65% 110M 0s 2024-03-04T18:06:46.165 INFO:teuthology.orchestra.run.smithi137.stderr: 60800K .......... .......... .......... .......... .......... 65% 110M 0s 2024-03-04T18:06:46.165 INFO:teuthology.orchestra.run.smithi137.stderr: 60850K .......... .......... .......... .......... .......... 65% 93.1M 0s 2024-03-04T18:06:46.165 INFO:teuthology.orchestra.run.smithi137.stderr: 60900K .......... .......... .......... .......... .......... 65% 118M 0s 2024-03-04T18:06:46.165 INFO:teuthology.orchestra.run.smithi137.stderr: 60950K .......... .......... .......... .......... .......... 65% 113M 0s 2024-03-04T18:06:46.165 INFO:teuthology.orchestra.run.smithi137.stderr: 61000K .......... .......... .......... .......... .......... 65% 114M 0s 2024-03-04T18:06:46.165 INFO:teuthology.orchestra.run.smithi137.stderr: 61050K .......... .......... .......... .......... .......... 65% 41.3M 0s 2024-03-04T18:06:46.165 INFO:teuthology.orchestra.run.smithi137.stderr: 61100K .......... .......... .......... .......... .......... 65% 113M 0s 2024-03-04T18:06:46.165 INFO:teuthology.orchestra.run.smithi137.stderr: 61150K .......... .......... .......... .......... .......... 65% 112M 0s 2024-03-04T18:06:46.165 INFO:teuthology.orchestra.run.smithi137.stderr: 61200K .......... .......... .......... .......... .......... 65% 115M 0s 2024-03-04T18:06:46.165 INFO:teuthology.orchestra.run.smithi137.stderr: 61250K .......... .......... .......... .......... .......... 65% 88.0M 0s 2024-03-04T18:06:46.166 INFO:teuthology.orchestra.run.smithi137.stderr: 61300K .......... .......... .......... .......... .......... 66% 106M 0s 2024-03-04T18:06:46.166 INFO:teuthology.orchestra.run.smithi137.stderr: 61350K .......... .......... .......... .......... .......... 66% 105M 0s 2024-03-04T18:06:46.166 INFO:teuthology.orchestra.run.smithi137.stderr: 61400K .......... .......... .......... .......... .......... 66% 85.4M 0s 2024-03-04T18:06:46.166 INFO:teuthology.orchestra.run.smithi137.stderr: 61450K .......... .......... .......... .......... .......... 66% 110M 0s 2024-03-04T18:06:46.166 INFO:teuthology.orchestra.run.smithi137.stderr: 61500K .......... .......... .......... .......... .......... 66% 114M 0s 2024-03-04T18:06:46.166 INFO:teuthology.orchestra.run.smithi137.stderr: 61550K .......... .......... .......... .......... .......... 66% 113M 0s 2024-03-04T18:06:46.166 INFO:teuthology.orchestra.run.smithi137.stderr: 61600K .......... .......... .......... .......... .......... 66% 116M 0s 2024-03-04T18:06:46.166 INFO:teuthology.orchestra.run.smithi137.stderr: 61650K .......... .......... .......... .......... .......... 66% 92.0M 0s 2024-03-04T18:06:46.166 INFO:teuthology.orchestra.run.smithi137.stderr: 61700K .......... .......... .......... .......... .......... 66% 111M 0s 2024-03-04T18:06:46.166 INFO:teuthology.orchestra.run.smithi137.stderr: 61750K .......... .......... .......... .......... .......... 66% 116M 0s 2024-03-04T18:06:46.166 INFO:teuthology.orchestra.run.smithi137.stderr: 61800K .......... .......... .......... .......... .......... 66% 114M 0s 2024-03-04T18:06:46.167 INFO:teuthology.orchestra.run.smithi137.stderr: 61850K .......... .......... .......... .......... .......... 66% 102M 0s 2024-03-04T18:06:46.167 INFO:teuthology.orchestra.run.smithi137.stderr: 61900K .......... .......... .......... .......... .......... 66% 116M 0s 2024-03-04T18:06:46.167 INFO:teuthology.orchestra.run.smithi137.stderr: 61950K .......... .......... .......... .......... .......... 66% 116M 0s 2024-03-04T18:06:46.167 INFO:teuthology.orchestra.run.smithi137.stderr: 62000K .......... .......... .......... .......... .......... 66% 114M 0s 2024-03-04T18:06:46.167 INFO:teuthology.orchestra.run.smithi137.stderr: 62050K .......... .......... .......... .......... .......... 66% 86.5M 0s 2024-03-04T18:06:46.167 INFO:teuthology.orchestra.run.smithi137.stderr: 62100K .......... .......... .......... .......... .......... 66% 117M 0s 2024-03-04T18:06:46.167 INFO:teuthology.orchestra.run.smithi137.stderr: 62150K .......... .......... .......... .......... .......... 66% 109M 0s 2024-03-04T18:06:46.167 INFO:teuthology.orchestra.run.smithi137.stderr: 62200K .......... .......... .......... .......... .......... 67% 110M 0s 2024-03-04T18:06:46.167 INFO:teuthology.orchestra.run.smithi137.stderr: 62250K .......... .......... .......... .......... .......... 67% 107M 0s 2024-03-04T18:06:46.167 INFO:teuthology.orchestra.run.smithi137.stderr: 62300K .......... .......... .......... .......... .......... 67% 119M 0s 2024-03-04T18:06:46.167 INFO:teuthology.orchestra.run.smithi137.stderr: 62350K .......... .......... .......... .......... .......... 67% 105M 0s 2024-03-04T18:06:46.167 INFO:teuthology.orchestra.run.smithi137.stderr: 62400K .......... .......... .......... .......... .......... 67% 116M 0s 2024-03-04T18:06:46.167 INFO:teuthology.orchestra.run.smithi137.stderr: 62450K .......... .......... .......... .......... .......... 67% 83.0M 0s 2024-03-04T18:06:46.168 INFO:teuthology.orchestra.run.smithi137.stderr: 62500K .......... .......... .......... .......... .......... 67% 109M 0s 2024-03-04T18:06:46.168 INFO:teuthology.orchestra.run.smithi137.stderr: 62550K .......... .......... .......... .......... .......... 67% 122M 0s 2024-03-04T18:06:46.168 INFO:teuthology.orchestra.run.smithi137.stderr: 62600K .......... .......... .......... .......... .......... 67% 113M 0s 2024-03-04T18:06:46.168 INFO:teuthology.orchestra.run.smithi137.stderr: 62650K .......... .......... .......... .......... .......... 67% 97.5M 0s 2024-03-04T18:06:46.168 INFO:teuthology.orchestra.run.smithi137.stderr: 62700K .......... .......... .......... .......... .......... 67% 115M 0s 2024-03-04T18:06:46.168 INFO:teuthology.orchestra.run.smithi137.stderr: 62750K .......... .......... .......... .......... .......... 67% 109M 0s 2024-03-04T18:06:46.168 INFO:teuthology.orchestra.run.smithi137.stderr: 62800K .......... .......... .......... .......... .......... 67% 114M 0s 2024-03-04T18:06:46.168 INFO:teuthology.orchestra.run.smithi137.stderr: 62850K .......... .......... .......... .......... .......... 67% 85.7M 0s 2024-03-04T18:06:46.168 INFO:teuthology.orchestra.run.smithi137.stderr: 62900K .......... .......... .......... .......... .......... 67% 115M 0s 2024-03-04T18:06:46.168 INFO:teuthology.orchestra.run.smithi137.stderr: 62950K .......... .......... .......... .......... .......... 67% 115M 0s 2024-03-04T18:06:46.168 INFO:teuthology.orchestra.run.smithi137.stderr: 63000K .......... .......... .......... .......... .......... 67% 117M 0s 2024-03-04T18:06:46.168 INFO:teuthology.orchestra.run.smithi137.stderr: 63050K .......... .......... .......... .......... .......... 67% 104M 0s 2024-03-04T18:06:46.168 INFO:teuthology.orchestra.run.smithi137.stderr: 63100K .......... .......... .......... .......... .......... 67% 110M 0s 2024-03-04T18:06:46.168 INFO:teuthology.orchestra.run.smithi137.stderr: 63150K .......... .......... .......... .......... .......... 68% 110M 0s 2024-03-04T18:06:46.169 INFO:teuthology.orchestra.run.smithi137.stderr: 63200K .......... .......... .......... .......... .......... 68% 121M 0s 2024-03-04T18:06:46.169 INFO:teuthology.orchestra.run.smithi137.stderr: 63250K .......... .......... .......... .......... .......... 68% 86.2M 0s 2024-03-04T18:06:46.169 INFO:teuthology.orchestra.run.smithi137.stderr: 63300K .......... .......... .......... .......... .......... 68% 109M 0s 2024-03-04T18:06:46.169 INFO:teuthology.orchestra.run.smithi137.stderr: 63350K .......... .......... .......... .......... .......... 68% 107M 0s 2024-03-04T18:06:46.169 INFO:teuthology.orchestra.run.smithi137.stderr: 63400K .......... .......... .......... .......... .......... 68% 108M 0s 2024-03-04T18:06:46.169 INFO:teuthology.orchestra.run.smithi137.stderr: 63450K .......... .......... .......... .......... .......... 68% 114M 0s 2024-03-04T18:06:46.169 INFO:teuthology.orchestra.run.smithi137.stderr: 63500K .......... .......... .......... .......... .......... 68% 112M 0s 2024-03-04T18:06:46.169 INFO:teuthology.orchestra.run.smithi137.stderr: 63550K .......... .......... .......... .......... .......... 68% 113M 0s 2024-03-04T18:06:46.169 INFO:teuthology.orchestra.run.smithi137.stderr: 63600K .......... .......... .......... .......... .......... 68% 104M 0s 2024-03-04T18:06:46.169 INFO:teuthology.orchestra.run.smithi137.stderr: 63650K .......... .......... .......... .......... .......... 68% 75.3M 0s 2024-03-04T18:06:46.169 INFO:teuthology.orchestra.run.smithi137.stderr: 63700K .......... .......... .......... .......... .......... 68% 118M 0s 2024-03-04T18:06:46.169 INFO:teuthology.orchestra.run.smithi137.stderr: 63750K .......... .......... .......... .......... .......... 68% 113M 0s 2024-03-04T18:06:46.169 INFO:teuthology.orchestra.run.smithi137.stderr: 63800K .......... .......... .......... .......... .......... 68% 110M 0s 2024-03-04T18:06:46.169 INFO:teuthology.orchestra.run.smithi137.stderr: 63850K .......... .......... .......... .......... .......... 68% 108M 0s 2024-03-04T18:06:46.170 INFO:teuthology.orchestra.run.smithi137.stderr: 63900K .......... .......... .......... .......... .......... 68% 122M 0s 2024-03-04T18:06:46.170 INFO:teuthology.orchestra.run.smithi137.stderr: 63950K .......... .......... .......... .......... .......... 68% 110M 0s 2024-03-04T18:06:46.170 INFO:teuthology.orchestra.run.smithi137.stderr: 64000K .......... .......... .......... .......... .......... 68% 97.3M 0s 2024-03-04T18:06:46.170 INFO:teuthology.orchestra.run.smithi137.stderr: 64050K .......... .......... .......... .......... .......... 68% 77.2M 0s 2024-03-04T18:06:46.170 INFO:teuthology.orchestra.run.smithi137.stderr: 64100K .......... .......... .......... .......... .......... 69% 67.7M 0s 2024-03-04T18:06:46.170 INFO:teuthology.orchestra.run.smithi137.stderr: 64150K .......... .......... .......... .......... .......... 69% 120M 0s 2024-03-04T18:06:46.170 INFO:teuthology.orchestra.run.smithi137.stderr: 64200K .......... .......... .......... .......... .......... 69% 113M 0s 2024-03-04T18:06:46.170 INFO:teuthology.orchestra.run.smithi137.stderr: 64250K .......... .......... .......... .......... .......... 69% 39.3M 0s 2024-03-04T18:06:46.170 INFO:teuthology.orchestra.run.smithi137.stderr: 64300K .......... .......... .......... .......... .......... 69% 52.5M 0s 2024-03-04T18:06:46.170 INFO:teuthology.orchestra.run.smithi137.stderr: 64350K .......... .......... .......... .......... .......... 69% 88.6M 0s 2024-03-04T18:06:46.170 INFO:teuthology.orchestra.run.smithi137.stderr: 64400K .......... .......... .......... .......... .......... 69% 112M 0s 2024-03-04T18:06:46.170 INFO:teuthology.orchestra.run.smithi137.stderr: 64450K .......... .......... .......... .......... .......... 69% 91.7M 0s 2024-03-04T18:06:46.170 INFO:teuthology.orchestra.run.smithi137.stderr: 64500K .......... .......... .......... .......... .......... 69% 110M 0s 2024-03-04T18:06:46.171 INFO:teuthology.orchestra.run.smithi137.stderr: 64550K .......... .......... .......... .......... .......... 69% 107M 0s 2024-03-04T18:06:46.171 INFO:teuthology.orchestra.run.smithi137.stderr: 64600K .......... .......... .......... .......... .......... 69% 123M 0s 2024-03-04T18:06:46.171 INFO:teuthology.orchestra.run.smithi137.stderr: 64650K .......... .......... .......... .......... .......... 69% 102M 0s 2024-03-04T18:06:46.171 INFO:teuthology.orchestra.run.smithi137.stderr: 64700K .......... .......... .......... .......... .......... 69% 113M 0s 2024-03-04T18:06:46.171 INFO:teuthology.orchestra.run.smithi137.stderr: 64750K .......... .......... .......... .......... .......... 69% 110M 0s 2024-03-04T18:06:46.171 INFO:teuthology.orchestra.run.smithi137.stderr: 64800K .......... .......... .......... .......... .......... 69% 108M 0s 2024-03-04T18:06:46.171 INFO:teuthology.orchestra.run.smithi137.stderr: 64850K .......... .......... .......... .......... .......... 69% 84.7M 0s 2024-03-04T18:06:46.171 INFO:teuthology.orchestra.run.smithi137.stderr: 64900K .......... .......... .......... .......... .......... 69% 114M 0s 2024-03-04T18:06:46.171 INFO:teuthology.orchestra.run.smithi137.stderr: 64950K .......... .......... .......... .......... .......... 69% 108M 0s 2024-03-04T18:06:46.171 INFO:teuthology.orchestra.run.smithi137.stderr: 65000K .......... .......... .......... .......... .......... 70% 106M 0s 2024-03-04T18:06:46.171 INFO:teuthology.orchestra.run.smithi137.stderr: 65050K .......... .......... .......... .......... .......... 70% 97.9M 0s 2024-03-04T18:06:46.171 INFO:teuthology.orchestra.run.smithi137.stderr: 65100K .......... .......... .......... .......... .......... 70% 131M 0s 2024-03-04T18:06:46.172 INFO:teuthology.orchestra.run.smithi137.stderr: 65150K .......... .......... .......... .......... .......... 70% 110M 0s 2024-03-04T18:06:46.172 INFO:teuthology.orchestra.run.smithi137.stderr: 65200K .......... .......... .......... .......... .......... 70% 105M 0s 2024-03-04T18:06:46.172 INFO:teuthology.orchestra.run.smithi137.stderr: 65250K .......... .......... .......... .......... .......... 70% 87.9M 0s 2024-03-04T18:06:46.172 INFO:teuthology.orchestra.run.smithi137.stderr: 65300K .......... .......... .......... .......... .......... 70% 77.7M 0s 2024-03-04T18:06:46.172 INFO:teuthology.orchestra.run.smithi137.stderr: 65350K .......... .......... .......... .......... .......... 70% 110M 0s 2024-03-04T18:06:46.172 INFO:teuthology.orchestra.run.smithi137.stderr: 65400K .......... .......... .......... .......... .......... 70% 109M 0s 2024-03-04T18:06:46.172 INFO:teuthology.orchestra.run.smithi137.stderr: 65450K .......... .......... .......... .......... .......... 70% 93.4M 0s 2024-03-04T18:06:46.172 INFO:teuthology.orchestra.run.smithi137.stderr: 65500K .......... .......... .......... .......... .......... 70% 112M 0s 2024-03-04T18:06:46.172 INFO:teuthology.orchestra.run.smithi137.stderr: 65550K .......... .......... .......... .......... .......... 70% 116M 0s 2024-03-04T18:06:46.172 INFO:teuthology.orchestra.run.smithi137.stderr: 65600K .......... .......... .......... .......... .......... 70% 114M 0s 2024-03-04T18:06:46.172 INFO:teuthology.orchestra.run.smithi137.stderr: 65650K .......... .......... .......... .......... .......... 70% 43.7M 0s 2024-03-04T18:06:46.173 INFO:teuthology.orchestra.run.smithi137.stderr: 65700K .......... .......... .......... .......... .......... 70% 110M 0s 2024-03-04T18:06:46.173 INFO:teuthology.orchestra.run.smithi137.stderr: 65750K .......... .......... .......... .......... .......... 70% 114M 0s 2024-03-04T18:06:46.173 INFO:teuthology.orchestra.run.smithi137.stderr: 65800K .......... .......... .......... .......... .......... 70% 117M 0s 2024-03-04T18:06:46.173 INFO:teuthology.orchestra.run.smithi137.stderr: 65850K .......... .......... .......... .......... .......... 70% 65.6M 0s 2024-03-04T18:06:46.173 INFO:teuthology.orchestra.run.smithi137.stderr: 65900K .......... .......... .......... .......... .......... 70% 115M 0s 2024-03-04T18:06:46.173 INFO:teuthology.orchestra.run.smithi137.stderr: 65950K .......... .......... .......... .......... .......... 71% 111M 0s 2024-03-04T18:06:46.173 INFO:teuthology.orchestra.run.smithi137.stderr: 66000K .......... .......... .......... .......... .......... 71% 116M 0s 2024-03-04T18:06:46.173 INFO:teuthology.orchestra.run.smithi137.stderr: 66050K .......... .......... .......... .......... .......... 71% 89.9M 0s 2024-03-04T18:06:46.173 INFO:teuthology.orchestra.run.smithi137.stderr: 66100K .......... .......... .......... .......... .......... 71% 119M 0s 2024-03-04T18:06:46.173 INFO:teuthology.orchestra.run.smithi137.stderr: 66150K .......... .......... .......... .......... .......... 71% 111M 0s 2024-03-04T18:06:46.173 INFO:teuthology.orchestra.run.smithi137.stderr: 66200K .......... .......... .......... .......... .......... 71% 103M 0s 2024-03-04T18:06:46.174 INFO:teuthology.orchestra.run.smithi137.stderr: 66250K .......... .......... .......... .......... .......... 71% 107M 0s 2024-03-04T18:06:46.174 INFO:teuthology.orchestra.run.smithi137.stderr: 66300K .......... .......... .......... .......... .......... 71% 117M 0s 2024-03-04T18:06:46.174 INFO:teuthology.orchestra.run.smithi137.stderr: 66350K .......... .......... .......... .......... .......... 71% 114M 0s 2024-03-04T18:06:46.174 INFO:teuthology.orchestra.run.smithi137.stderr: 66400K .......... .......... .......... .......... .......... 71% 112M 0s 2024-03-04T18:06:46.174 INFO:teuthology.orchestra.run.smithi137.stderr: 66450K .......... .......... .......... .......... .......... 71% 89.9M 0s 2024-03-04T18:06:46.174 INFO:teuthology.orchestra.run.smithi137.stderr: 66500K .......... .......... .......... .......... .......... 71% 117M 0s 2024-03-04T18:06:46.174 INFO:teuthology.orchestra.run.smithi137.stderr: 66550K .......... .......... .......... .......... .......... 71% 113M 0s 2024-03-04T18:06:46.174 INFO:teuthology.orchestra.run.smithi137.stderr: 66600K .......... .......... .......... .......... .......... 71% 107M 0s 2024-03-04T18:06:46.174 INFO:teuthology.orchestra.run.smithi137.stderr: 66650K .......... .......... .......... .......... .......... 71% 97.7M 0s 2024-03-04T18:06:46.174 INFO:teuthology.orchestra.run.smithi137.stderr: 66700K .......... .......... .......... .......... .......... 71% 92.5M 0s 2024-03-04T18:06:46.174 INFO:teuthology.orchestra.run.smithi137.stderr: 66750K .......... .......... .......... .......... .......... 71% 100M 0s 2024-03-04T18:06:46.174 INFO:teuthology.orchestra.run.smithi137.stderr: 66800K .......... .......... .......... .......... .......... 71% 115M 0s 2024-03-04T18:06:46.175 INFO:teuthology.orchestra.run.smithi137.stderr: 66850K .......... .......... .......... .......... .......... 72% 91.4M 0s 2024-03-04T18:06:46.175 INFO:teuthology.orchestra.run.smithi137.stderr: 66900K .......... .......... .......... .......... .......... 72% 114M 0s 2024-03-04T18:06:46.175 INFO:teuthology.orchestra.run.smithi137.stderr: 66950K .......... .......... .......... .......... .......... 72% 114M 0s 2024-03-04T18:06:46.175 INFO:teuthology.orchestra.run.smithi137.stderr: 67000K .......... .......... .......... .......... .......... 72% 117M 0s 2024-03-04T18:06:46.175 INFO:teuthology.orchestra.run.smithi137.stderr: 67050K .......... .......... .......... .......... .......... 72% 107M 0s 2024-03-04T18:06:46.175 INFO:teuthology.orchestra.run.smithi137.stderr: 67100K .......... .......... .......... .......... .......... 72% 114M 0s 2024-03-04T18:06:46.175 INFO:teuthology.orchestra.run.smithi137.stderr: 67150K .......... .......... .......... .......... .......... 72% 117M 0s 2024-03-04T18:06:46.175 INFO:teuthology.orchestra.run.smithi137.stderr: 67200K .......... .......... .......... .......... .......... 72% 109M 0s 2024-03-04T18:06:46.176 INFO:teuthology.orchestra.run.smithi137.stderr: 67250K .......... .......... .......... .......... .......... 72% 71.9M 0s 2024-03-04T18:06:46.176 INFO:teuthology.orchestra.run.smithi137.stderr: 67300K .......... .......... .......... .......... .......... 72% 162M 0s 2024-03-04T18:06:46.176 INFO:teuthology.orchestra.run.smithi137.stderr: 67350K .......... .......... .......... .......... .......... 72% 116M 0s 2024-03-04T18:06:46.176 INFO:teuthology.orchestra.run.smithi137.stderr: 67400K .......... .......... .......... .......... .......... 72% 112M 0s 2024-03-04T18:06:46.176 INFO:teuthology.orchestra.run.smithi137.stderr: 67450K .......... .......... .......... .......... .......... 72% 111M 0s 2024-03-04T18:06:46.176 INFO:teuthology.orchestra.run.smithi137.stderr: 67500K .......... .......... .......... .......... .......... 72% 105M 0s 2024-03-04T18:06:46.176 INFO:teuthology.orchestra.run.smithi137.stderr: 67550K .......... .......... .......... .......... .......... 72% 117M 0s 2024-03-04T18:06:46.176 INFO:teuthology.orchestra.run.smithi137.stderr: 67600K .......... .......... .......... .......... .......... 72% 111M 0s 2024-03-04T18:06:46.176 INFO:teuthology.orchestra.run.smithi137.stderr: 67650K .......... .......... .......... .......... .......... 72% 89.2M 0s 2024-03-04T18:06:46.176 INFO:teuthology.orchestra.run.smithi137.stderr: 67700K .......... .......... .......... .......... .......... 72% 114M 0s 2024-03-04T18:06:46.177 INFO:teuthology.orchestra.run.smithi137.stderr: 67750K .......... .......... .......... .......... .......... 72% 113M 0s 2024-03-04T18:06:46.177 INFO:teuthology.orchestra.run.smithi137.stderr: 67800K .......... .......... .......... .......... .......... 73% 114M 0s 2024-03-04T18:06:46.177 INFO:teuthology.orchestra.run.smithi137.stderr: 67850K .......... .......... .......... .......... .......... 73% 103M 0s 2024-03-04T18:06:46.177 INFO:teuthology.orchestra.run.smithi137.stderr: 67900K .......... .......... .......... .......... .......... 73% 119M 0s 2024-03-04T18:06:46.177 INFO:teuthology.orchestra.run.smithi137.stderr: 67950K .......... .......... .......... .......... .......... 73% 115M 0s 2024-03-04T18:06:46.177 INFO:teuthology.orchestra.run.smithi137.stderr: 68000K .......... .......... .......... .......... .......... 73% 111M 0s 2024-03-04T18:06:46.177 INFO:teuthology.orchestra.run.smithi137.stderr: 68050K .......... .......... .......... .......... .......... 73% 86.8M 0s 2024-03-04T18:06:46.177 INFO:teuthology.orchestra.run.smithi137.stderr: 68100K .......... .......... .......... .......... .......... 73% 108M 0s 2024-03-04T18:06:46.177 INFO:teuthology.orchestra.run.smithi137.stderr: 68150K .......... .......... .......... .......... .......... 73% 116M 0s 2024-03-04T18:06:46.177 INFO:teuthology.orchestra.run.smithi137.stderr: 68200K .......... .......... .......... .......... .......... 73% 110M 0s 2024-03-04T18:06:46.177 INFO:teuthology.orchestra.run.smithi137.stderr: 68250K .......... .......... .......... .......... .......... 73% 112M 0s 2024-03-04T18:06:46.177 INFO:teuthology.orchestra.run.smithi137.stderr: 68300K .......... .......... .......... .......... .......... 73% 112M 0s 2024-03-04T18:06:46.177 INFO:teuthology.orchestra.run.smithi137.stderr: 68350K .......... .......... .......... .......... .......... 73% 119M 0s 2024-03-04T18:06:46.177 INFO:teuthology.orchestra.run.smithi137.stderr: 68400K .......... .......... .......... .......... .......... 73% 108M 0s 2024-03-04T18:06:46.178 INFO:teuthology.orchestra.run.smithi137.stderr: 68450K .......... .......... .......... .......... .......... 73% 87.4M 0s 2024-03-04T18:06:46.178 INFO:teuthology.orchestra.run.smithi137.stderr: 68500K .......... .......... .......... .......... .......... 73% 119M 0s 2024-03-04T18:06:46.178 INFO:teuthology.orchestra.run.smithi137.stderr: 68550K .......... .......... .......... .......... .......... 73% 116M 0s 2024-03-04T18:06:46.178 INFO:teuthology.orchestra.run.smithi137.stderr: 68600K .......... .......... .......... .......... .......... 73% 112M 0s 2024-03-04T18:06:46.178 INFO:teuthology.orchestra.run.smithi137.stderr: 68650K .......... .......... .......... .......... .......... 73% 108M 0s 2024-03-04T18:06:46.178 INFO:teuthology.orchestra.run.smithi137.stderr: 68700K .......... .......... .......... .......... .......... 74% 108M 0s 2024-03-04T18:06:46.178 INFO:teuthology.orchestra.run.smithi137.stderr: 68750K .......... .......... .......... .......... .......... 74% 110M 0s 2024-03-04T18:06:46.178 INFO:teuthology.orchestra.run.smithi137.stderr: 68800K .......... .......... .......... .......... .......... 74% 117M 0s 2024-03-04T18:06:46.178 INFO:teuthology.orchestra.run.smithi137.stderr: 68850K .......... .......... .......... .......... .......... 74% 88.7M 0s 2024-03-04T18:06:46.178 INFO:teuthology.orchestra.run.smithi137.stderr: 68900K .......... .......... .......... .......... .......... 74% 112M 0s 2024-03-04T18:06:46.178 INFO:teuthology.orchestra.run.smithi137.stderr: 68950K .......... .......... .......... .......... .......... 74% 112M 0s 2024-03-04T18:06:46.178 INFO:teuthology.orchestra.run.smithi137.stderr: 69000K .......... .......... .......... .......... .......... 74% 114M 0s 2024-03-04T18:06:46.178 INFO:teuthology.orchestra.run.smithi137.stderr: 69050K .......... .......... .......... .......... .......... 74% 108M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69100K .......... .......... .......... .......... .......... 74% 117M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69150K .......... .......... .......... .......... .......... 74% 113M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69200K .......... .......... .......... .......... .......... 74% 119M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69250K .......... .......... .......... .......... .......... 74% 91.4M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69300K .......... .......... .......... .......... .......... 74% 116M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69350K .......... .......... .......... .......... .......... 74% 105M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69400K .......... .......... .......... .......... .......... 74% 124M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69450K .......... .......... .......... .......... .......... 74% 103M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69500K .......... .......... .......... .......... .......... 74% 115M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69550K .......... .......... .......... .......... .......... 74% 117M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69600K .......... .......... .......... .......... .......... 74% 110M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69650K .......... .......... .......... .......... .......... 75% 96.0M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69700K .......... .......... .......... .......... .......... 75% 111M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69750K .......... .......... .......... .......... .......... 75% 110M 0s 2024-03-04T18:06:46.179 INFO:teuthology.orchestra.run.smithi137.stderr: 69800K .......... .......... .......... .......... .......... 75% 119M 0s 2024-03-04T18:06:46.180 INFO:teuthology.orchestra.run.smithi137.stderr: 69850K .......... .......... .......... .......... .......... 75% 107M 0s 2024-03-04T18:06:46.180 INFO:teuthology.orchestra.run.smithi137.stderr: 69900K .......... .......... .......... .......... .......... 75% 115M 0s 2024-03-04T18:06:46.180 INFO:teuthology.orchestra.run.smithi137.stderr: 69950K .......... .......... .......... .......... .......... 75% 78.3M 0s 2024-03-04T18:06:46.180 INFO:teuthology.orchestra.run.smithi137.stderr: 70000K .......... .......... .......... .......... .......... 75% 164M 0s 2024-03-04T18:06:46.180 INFO:teuthology.orchestra.run.smithi137.stderr: 70050K .......... .......... .......... .......... .......... 75% 82.0M 0s 2024-03-04T18:06:46.180 INFO:teuthology.orchestra.run.smithi137.stderr: 70100K .......... .......... .......... .......... .......... 75% 131M 0s 2024-03-04T18:06:46.180 INFO:teuthology.orchestra.run.smithi137.stderr: 70150K .......... .......... .......... .......... .......... 75% 127M 0s 2024-03-04T18:06:46.180 INFO:teuthology.orchestra.run.smithi137.stderr: 70200K .......... .......... .......... .......... .......... 75% 122M 0s 2024-03-04T18:06:46.180 INFO:teuthology.orchestra.run.smithi137.stderr: 70250K .......... .......... .......... .......... .......... 75% 99.0M 0s 2024-03-04T18:06:46.180 INFO:teuthology.orchestra.run.smithi137.stderr: 70300K .......... .......... .......... .......... .......... 75% 116M 0s 2024-03-04T18:06:46.180 INFO:teuthology.orchestra.run.smithi137.stderr: 70350K .......... .......... .......... .......... .......... 75% 108M 0s 2024-03-04T18:06:46.180 INFO:teuthology.orchestra.run.smithi137.stderr: 70400K .......... .......... .......... .......... .......... 75% 123M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 70450K .......... .......... .......... .......... .......... 75% 86.7M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 70500K .......... .......... .......... .......... .......... 75% 114M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 70550K .......... .......... .......... .......... .......... 75% 100M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 70600K .......... .......... .......... .......... .......... 76% 119M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 70650K .......... .......... .......... .......... .......... 76% 97.0M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 70700K .......... .......... .......... .......... .......... 76% 111M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 70750K .......... .......... .......... .......... .......... 76% 117M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 70800K .......... .......... .......... .......... .......... 76% 45.1M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 70850K .......... .......... .......... .......... .......... 76% 87.2M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 70900K .......... .......... .......... .......... .......... 76% 128M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 70950K .......... .......... .......... .......... .......... 76% 117M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 71000K .......... .......... .......... .......... .......... 76% 112M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 71050K .......... .......... .......... .......... .......... 76% 105M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 71100K .......... .......... .......... .......... .......... 76% 111M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 71150K .......... .......... .......... .......... .......... 76% 108M 0s 2024-03-04T18:06:46.181 INFO:teuthology.orchestra.run.smithi137.stderr: 71200K .......... .......... .......... .......... .......... 76% 112M 0s 2024-03-04T18:06:46.182 INFO:teuthology.orchestra.run.smithi137.stderr: 71250K .......... .......... .......... .......... .......... 76% 102M 0s 2024-03-04T18:06:46.182 INFO:teuthology.orchestra.run.smithi137.stderr: 71300K .......... .......... .......... .......... .......... 76% 107M 0s 2024-03-04T18:06:46.182 INFO:teuthology.orchestra.run.smithi137.stderr: 71350K .......... .......... .......... .......... .......... 76% 59.1M 0s 2024-03-04T18:06:46.182 INFO:teuthology.orchestra.run.smithi137.stderr: 71400K .......... .......... .......... .......... .......... 76% 84.6M 0s 2024-03-04T18:06:46.182 INFO:teuthology.orchestra.run.smithi137.stderr: 71450K .......... .......... .......... .......... .......... 76% 62.6M 0s 2024-03-04T18:06:46.182 INFO:teuthology.orchestra.run.smithi137.stderr: 71500K .......... .......... .......... .......... .......... 77% 173M 0s 2024-03-04T18:06:46.182 INFO:teuthology.orchestra.run.smithi137.stderr: 71550K .......... .......... .......... .......... .......... 77% 171M 0s 2024-03-04T18:06:46.182 INFO:teuthology.orchestra.run.smithi137.stderr: 71600K .......... .......... .......... .......... .......... 77% 121M 0s 2024-03-04T18:06:46.182 INFO:teuthology.orchestra.run.smithi137.stderr: 71650K .......... .......... .......... .......... .......... 77% 89.7M 0s 2024-03-04T18:06:46.182 INFO:teuthology.orchestra.run.smithi137.stderr: 71700K .......... .......... .......... .......... .......... 77% 113M 0s 2024-03-04T18:06:46.182 INFO:teuthology.orchestra.run.smithi137.stderr: 71750K .......... .......... .......... .......... .......... 77% 115M 0s 2024-03-04T18:06:46.182 INFO:teuthology.orchestra.run.smithi137.stderr: 71800K .......... .......... .......... .......... .......... 77% 113M 0s 2024-03-04T18:06:46.182 INFO:teuthology.orchestra.run.smithi137.stderr: 71850K .......... .......... .......... .......... .......... 77% 106M 0s 2024-03-04T18:06:46.182 INFO:teuthology.orchestra.run.smithi137.stderr: 71900K .......... .......... .......... .......... .......... 77% 99.8M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 71950K .......... .......... .......... .......... .......... 77% 109M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72000K .......... .......... .......... .......... .......... 77% 111M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72050K .......... .......... .......... .......... .......... 77% 90.5M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72100K .......... .......... .......... .......... .......... 77% 114M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72150K .......... .......... .......... .......... .......... 77% 118M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72200K .......... .......... .......... .......... .......... 77% 105M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72250K .......... .......... .......... .......... .......... 77% 108M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72300K .......... .......... .......... .......... .......... 77% 120M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72350K .......... .......... .......... .......... .......... 77% 102M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72400K .......... .......... .......... .......... .......... 77% 110M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72450K .......... .......... .......... .......... .......... 78% 93.0M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72500K .......... .......... .......... .......... .......... 78% 110M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72550K .......... .......... .......... .......... .......... 78% 110M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72600K .......... .......... .......... .......... .......... 78% 119M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72650K .......... .......... .......... .......... .......... 78% 102M 0s 2024-03-04T18:06:46.183 INFO:teuthology.orchestra.run.smithi137.stderr: 72700K .......... .......... .......... .......... .......... 78% 109M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 72750K .......... .......... .......... .......... .......... 78% 107M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 72800K .......... .......... .......... .......... .......... 78% 116M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 72850K .......... .......... .......... .......... .......... 78% 88.2M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 72900K .......... .......... .......... .......... .......... 78% 109M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 72950K .......... .......... .......... .......... .......... 78% 114M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 73000K .......... .......... .......... .......... .......... 78% 108M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 73050K .......... .......... .......... .......... .......... 78% 115M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 73100K .......... .......... .......... .......... .......... 78% 115M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 73150K .......... .......... .......... .......... .......... 78% 111M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 73200K .......... .......... .......... .......... .......... 78% 106M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 73250K .......... .......... .......... .......... .......... 78% 83.5M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 73300K .......... .......... .......... .......... .......... 78% 122M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 73350K .......... .......... .......... .......... .......... 79% 107M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 73400K .......... .......... .......... .......... .......... 79% 104M 0s 2024-03-04T18:06:46.184 INFO:teuthology.orchestra.run.smithi137.stderr: 73450K .......... .......... .......... .......... .......... 79% 104M 0s 2024-03-04T18:06:46.185 INFO:teuthology.orchestra.run.smithi137.stderr: 73500K .......... .......... .......... .......... .......... 79% 117M 0s 2024-03-04T18:06:46.185 INFO:teuthology.orchestra.run.smithi137.stderr: 73550K .......... .......... .......... .......... .......... 79% 111M 0s 2024-03-04T18:06:46.185 INFO:teuthology.orchestra.run.smithi137.stderr: 73600K .......... .......... .......... .......... .......... 79% 111M 0s 2024-03-04T18:06:46.185 INFO:teuthology.orchestra.run.smithi137.stderr: 73650K .......... .......... .......... .......... .......... 79% 92.8M 0s 2024-03-04T18:06:46.185 INFO:teuthology.orchestra.run.smithi137.stderr: 73700K .......... .......... .......... .......... .......... 79% 102M 0s 2024-03-04T18:06:46.185 INFO:teuthology.orchestra.run.smithi137.stderr: 73750K .......... .......... .......... .......... .......... 79% 125M 0s 2024-03-04T18:06:46.185 INFO:teuthology.orchestra.run.smithi137.stderr: 73800K .......... .......... .......... .......... .......... 79% 111M 0s 2024-03-04T18:06:46.185 INFO:teuthology.orchestra.run.smithi137.stderr: 73850K .......... .......... .......... .......... .......... 79% 107M 0s 2024-03-04T18:06:46.185 INFO:teuthology.orchestra.run.smithi137.stderr: 73900K .......... .......... .......... .......... .......... 79% 109M 0s 2024-03-04T18:06:46.185 INFO:teuthology.orchestra.run.smithi137.stderr: 73950K .......... .......... .......... .......... .......... 79% 111M 0s 2024-03-04T18:06:46.185 INFO:teuthology.orchestra.run.smithi137.stderr: 74000K .......... .......... .......... .......... .......... 79% 117M 0s 2024-03-04T18:06:46.185 INFO:teuthology.orchestra.run.smithi137.stderr: 74050K .......... .......... .......... .......... .......... 79% 91.2M 0s 2024-03-04T18:06:46.185 INFO:teuthology.orchestra.run.smithi137.stderr: 74100K .......... .......... .......... .......... .......... 79% 89.2M 0s 2024-03-04T18:06:46.185 INFO:teuthology.orchestra.run.smithi137.stderr: 74150K .......... .......... .......... .......... .......... 79% 46.5M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74200K .......... .......... .......... .......... .......... 79% 114M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74250K .......... .......... .......... .......... .......... 79% 109M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74300K .......... .......... .......... .......... .......... 80% 44.0M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74350K .......... .......... .......... .......... .......... 80% 114M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74400K .......... .......... .......... .......... .......... 80% 112M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74450K .......... .......... .......... .......... .......... 80% 91.9M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74500K .......... .......... .......... .......... .......... 80% 105M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74550K .......... .......... .......... .......... .......... 80% 125M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74600K .......... .......... .......... .......... .......... 80% 112M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74650K .......... .......... .......... .......... .......... 80% 110M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74700K .......... .......... .......... .......... .......... 80% 113M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74750K .......... .......... .......... .......... .......... 80% 114M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74800K .......... .......... .......... .......... .......... 80% 105M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74850K .......... .......... .......... .......... .......... 80% 87.0M 0s 2024-03-04T18:06:46.186 INFO:teuthology.orchestra.run.smithi137.stderr: 74900K .......... .......... .......... .......... .......... 80% 118M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 74950K .......... .......... .......... .......... .......... 80% 109M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75000K .......... .......... .......... .......... .......... 80% 114M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75050K .......... .......... .......... .......... .......... 80% 93.3M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75100K .......... .......... .......... .......... .......... 80% 119M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75150K .......... .......... .......... .......... .......... 80% 98.0M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75200K .......... .......... .......... .......... .......... 81% 109M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75250K .......... .......... .......... .......... .......... 81% 98.7M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75300K .......... .......... .......... .......... .......... 81% 110M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75350K .......... .......... .......... .......... .......... 81% 121M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75400K .......... .......... .......... .......... .......... 81% 97.0M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75450K .......... .......... .......... .......... .......... 81% 94.8M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75500K .......... .......... .......... .......... .......... 81% 109M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75550K .......... .......... .......... .......... .......... 81% 76.0M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75600K .......... .......... .......... .......... .......... 81% 108M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75650K .......... .......... .......... .......... .......... 81% 88.5M 0s 2024-03-04T18:06:46.187 INFO:teuthology.orchestra.run.smithi137.stderr: 75700K .......... .......... .......... .......... .......... 81% 118M 0s 2024-03-04T18:06:46.188 INFO:teuthology.orchestra.run.smithi137.stderr: 75750K .......... .......... .......... .......... .......... 81% 105M 0s 2024-03-04T18:06:46.188 INFO:teuthology.orchestra.run.smithi137.stderr: 75800K .......... .......... .......... .......... .......... 81% 119M 0s 2024-03-04T18:06:46.188 INFO:teuthology.orchestra.run.smithi137.stderr: 75850K .......... .......... .......... .......... .......... 81% 109M 0s 2024-03-04T18:06:46.188 INFO:teuthology.orchestra.run.smithi137.stderr: 75900K .......... .......... .......... .......... .......... 81% 114M 0s 2024-03-04T18:06:46.188 INFO:teuthology.orchestra.run.smithi137.stderr: 75950K .......... .......... .......... .......... .......... 81% 106M 0s 2024-03-04T18:06:46.188 INFO:teuthology.orchestra.run.smithi137.stderr: 76000K .......... .......... .......... .......... .......... 81% 112M 0s 2024-03-04T18:06:46.188 INFO:teuthology.orchestra.run.smithi137.stderr: 76050K .......... .......... .......... .......... .......... 81% 80.8M 0s 2024-03-04T18:06:46.188 INFO:teuthology.orchestra.run.smithi137.stderr: 76100K .......... .......... .......... .......... .......... 81% 113M 0s 2024-03-04T18:06:46.188 INFO:teuthology.orchestra.run.smithi137.stderr: 76150K .......... .......... .......... .......... .......... 82% 123M 0s 2024-03-04T18:06:46.188 INFO:teuthology.orchestra.run.smithi137.stderr: 76200K .......... .......... .......... .......... .......... 82% 107M 0s 2024-03-04T18:06:46.188 INFO:teuthology.orchestra.run.smithi137.stderr: 76250K .......... .......... .......... .......... .......... 82% 98.4M 0s 2024-03-04T18:06:46.188 INFO:teuthology.orchestra.run.smithi137.stderr: 76300K .......... .......... .......... .......... .......... 82% 122M 0s 2024-03-04T18:06:46.188 INFO:teuthology.orchestra.run.smithi137.stderr: 76350K .......... .......... .......... .......... .......... 82% 112M 0s 2024-03-04T18:06:46.189 INFO:teuthology.orchestra.run.smithi137.stderr: 76400K .......... .......... .......... .......... .......... 82% 104M 0s 2024-03-04T18:06:46.189 INFO:teuthology.orchestra.run.smithi137.stderr: 76450K .......... .......... .......... .......... .......... 82% 87.6M 0s 2024-03-04T18:06:46.189 INFO:teuthology.orchestra.run.smithi137.stderr: 76500K .......... .......... .......... .......... .......... 82% 112M 0s 2024-03-04T18:06:46.189 INFO:teuthology.orchestra.run.smithi137.stderr: 76550K .......... .......... .......... .......... .......... 82% 115M 0s 2024-03-04T18:06:46.189 INFO:teuthology.orchestra.run.smithi137.stderr: 76600K .......... .......... .......... .......... .......... 82% 112M 0s 2024-03-04T18:06:46.189 INFO:teuthology.orchestra.run.smithi137.stderr: 76650K .......... .......... .......... .......... .......... 82% 101M 0s 2024-03-04T18:06:46.189 INFO:teuthology.orchestra.run.smithi137.stderr: 76700K .......... .......... .......... .......... .......... 82% 103M 0s 2024-03-04T18:06:46.189 INFO:teuthology.orchestra.run.smithi137.stderr: 76750K .......... .......... .......... .......... .......... 82% 113M 0s 2024-03-04T18:06:46.189 INFO:teuthology.orchestra.run.smithi137.stderr: 76800K .......... .......... .......... .......... .......... 82% 118M 0s 2024-03-04T18:06:46.189 INFO:teuthology.orchestra.run.smithi137.stderr: 76850K .......... .......... .......... .......... .......... 82% 91.0M 0s 2024-03-04T18:06:46.189 INFO:teuthology.orchestra.run.smithi137.stderr: 76900K .......... .......... .......... .......... .......... 82% 112M 0s 2024-03-04T18:06:46.189 INFO:teuthology.orchestra.run.smithi137.stderr: 76950K .......... .......... .......... .......... .......... 82% 121M 0s 2024-03-04T18:06:46.189 INFO:teuthology.orchestra.run.smithi137.stderr: 77000K .......... .......... .......... .......... .......... 82% 109M 0s 2024-03-04T18:06:46.189 INFO:teuthology.orchestra.run.smithi137.stderr: 77050K .......... .......... .......... .......... .......... 82% 109M 0s 2024-03-04T18:06:46.190 INFO:teuthology.orchestra.run.smithi137.stderr: 77100K .......... .......... .......... .......... .......... 83% 55.9M 0s 2024-03-04T18:06:46.190 INFO:teuthology.orchestra.run.smithi137.stderr: 77150K .......... .......... .......... .......... .......... 83% 87.0M 0s 2024-03-04T18:06:46.190 INFO:teuthology.orchestra.run.smithi137.stderr: 77200K .......... .......... .......... .......... .......... 83% 112M 0s 2024-03-04T18:06:46.190 INFO:teuthology.orchestra.run.smithi137.stderr: 77250K .......... .......... .......... .......... .......... 83% 89.4M 0s 2024-03-04T18:06:46.190 INFO:teuthology.orchestra.run.smithi137.stderr: 77300K .......... .......... .......... .......... .......... 83% 91.6M 0s 2024-03-04T18:06:46.190 INFO:teuthology.orchestra.run.smithi137.stderr: 77350K .......... .......... .......... .......... .......... 83% 104M 0s 2024-03-04T18:06:46.190 INFO:teuthology.orchestra.run.smithi137.stderr: 77400K .......... .......... .......... .......... .......... 83% 117M 0s 2024-03-04T18:06:46.190 INFO:teuthology.orchestra.run.smithi137.stderr: 77450K .......... .......... .......... .......... .......... 83% 109M 0s 2024-03-04T18:06:46.190 INFO:teuthology.orchestra.run.smithi137.stderr: 77500K .......... .......... .......... .......... .......... 83% 115M 0s 2024-03-04T18:06:46.190 INFO:teuthology.orchestra.run.smithi137.stderr: 77550K .......... .......... .......... .......... .......... 83% 106M 0s 2024-03-04T18:06:46.190 INFO:teuthology.orchestra.run.smithi137.stderr: 77600K .......... .......... .......... .......... .......... 83% 125M 0s 2024-03-04T18:06:46.190 INFO:teuthology.orchestra.run.smithi137.stderr: 77650K .......... .......... .......... .......... .......... 83% 88.5M 0s 2024-03-04T18:06:46.190 INFO:teuthology.orchestra.run.smithi137.stderr: 77700K .......... .......... .......... .......... .......... 83% 118M 0s 2024-03-04T18:06:46.190 INFO:teuthology.orchestra.run.smithi137.stderr: 77750K .......... .......... .......... .......... .......... 83% 111M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 77800K .......... .......... .......... .......... .......... 83% 106M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 77850K .......... .......... .......... .......... .......... 83% 107M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 77900K .......... .......... .......... .......... .......... 83% 109M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 77950K .......... .......... .......... .......... .......... 83% 119M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 78000K .......... .......... .......... .......... .......... 84% 114M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 78050K .......... .......... .......... .......... .......... 84% 93.2M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 78100K .......... .......... .......... .......... .......... 84% 71.6M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 78150K .......... .......... .......... .......... .......... 84% 99.8M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 78200K .......... .......... .......... .......... .......... 84% 120M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 78250K .......... .......... .......... .......... .......... 84% 90.6M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 78300K .......... .......... .......... .......... .......... 84% 102M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 78350K .......... .......... .......... .......... .......... 84% 102M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 78400K .......... .......... .......... .......... .......... 84% 121M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 78450K .......... .......... .......... .......... .......... 84% 88.5M 0s 2024-03-04T18:06:46.191 INFO:teuthology.orchestra.run.smithi137.stderr: 78500K .......... .......... .......... .......... .......... 84% 104M 0s 2024-03-04T18:06:46.192 INFO:teuthology.orchestra.run.smithi137.stderr: 78550K .......... .......... .......... .......... .......... 84% 108M 0s 2024-03-04T18:06:46.192 INFO:teuthology.orchestra.run.smithi137.stderr: 78600K .......... .......... .......... .......... .......... 84% 111M 0s 2024-03-04T18:06:46.192 INFO:teuthology.orchestra.run.smithi137.stderr: 78650K .......... .......... .......... .......... .......... 84% 107M 0s 2024-03-04T18:06:46.192 INFO:teuthology.orchestra.run.smithi137.stderr: 78700K .......... .......... .......... .......... .......... 84% 109M 0s 2024-03-04T18:06:46.192 INFO:teuthology.orchestra.run.smithi137.stderr: 78750K .......... .......... .......... .......... .......... 84% 109M 0s 2024-03-04T18:06:46.192 INFO:teuthology.orchestra.run.smithi137.stderr: 78800K .......... .......... .......... .......... .......... 84% 115M 0s 2024-03-04T18:06:46.192 INFO:teuthology.orchestra.run.smithi137.stderr: 78850K .......... .......... .......... .......... .......... 84% 91.4M 0s 2024-03-04T18:06:46.192 INFO:teuthology.orchestra.run.smithi137.stderr: 78900K .......... .......... .......... .......... .......... 84% 113M 0s 2024-03-04T18:06:46.192 INFO:teuthology.orchestra.run.smithi137.stderr: 78950K .......... .......... .......... .......... .......... 85% 108M 0s 2024-03-04T18:06:46.192 INFO:teuthology.orchestra.run.smithi137.stderr: 79000K .......... .......... .......... .......... .......... 85% 86.4M 0s 2024-03-04T18:06:46.192 INFO:teuthology.orchestra.run.smithi137.stderr: 79050K .......... .......... .......... .......... .......... 85% 110M 0s 2024-03-04T18:06:46.192 INFO:teuthology.orchestra.run.smithi137.stderr: 79100K .......... .......... .......... .......... .......... 85% 118M 0s 2024-03-04T18:06:46.192 INFO:teuthology.orchestra.run.smithi137.stderr: 79150K .......... .......... .......... .......... .......... 85% 108M 0s 2024-03-04T18:06:46.192 INFO:teuthology.orchestra.run.smithi137.stderr: 79200K .......... .......... .......... .......... .......... 85% 111M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79250K .......... .......... .......... .......... .......... 85% 92.3M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79300K .......... .......... .......... .......... .......... 85% 110M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79350K .......... .......... .......... .......... .......... 85% 113M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79400K .......... .......... .......... .......... .......... 85% 118M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79450K .......... .......... .......... .......... .......... 85% 107M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79500K .......... .......... .......... .......... .......... 85% 107M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79550K .......... .......... .......... .......... .......... 85% 121M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79600K .......... .......... .......... .......... .......... 85% 114M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79650K .......... .......... .......... .......... .......... 85% 82.0M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79700K .......... .......... .......... .......... .......... 85% 122M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79750K .......... .......... .......... .......... .......... 85% 89.6M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79800K .......... .......... .......... .......... .......... 85% 119M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79850K .......... .......... .......... .......... .......... 86% 98.3M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79900K .......... .......... .......... .......... .......... 86% 109M 0s 2024-03-04T18:06:46.193 INFO:teuthology.orchestra.run.smithi137.stderr: 79950K .......... .......... .......... .......... .......... 86% 113M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80000K .......... .......... .......... .......... .......... 86% 123M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80050K .......... .......... .......... .......... .......... 86% 75.9M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80100K .......... .......... .......... .......... .......... 86% 113M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80150K .......... .......... .......... .......... .......... 86% 117M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80200K .......... .......... .......... .......... .......... 86% 103M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80250K .......... .......... .......... .......... .......... 86% 114M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80300K .......... .......... .......... .......... .......... 86% 114M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80350K .......... .......... .......... .......... .......... 86% 119M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80400K .......... .......... .......... .......... .......... 86% 113M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80450K .......... .......... .......... .......... .......... 86% 90.8M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80500K .......... .......... .......... .......... .......... 86% 117M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80550K .......... .......... .......... .......... .......... 86% 59.9M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80600K .......... .......... .......... .......... .......... 86% 74.1M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80650K .......... .......... .......... .......... .......... 86% 103M 0s 2024-03-04T18:06:46.194 INFO:teuthology.orchestra.run.smithi137.stderr: 80700K .......... .......... .......... .......... .......... 86% 127M 0s 2024-03-04T18:06:46.195 INFO:teuthology.orchestra.run.smithi137.stderr: 80750K .......... .......... .......... .......... .......... 86% 70.9M 0s 2024-03-04T18:06:46.195 INFO:teuthology.orchestra.run.smithi137.stderr: 80800K .......... .......... .......... .......... .......... 87% 75.9M 0s 2024-03-04T18:06:46.195 INFO:teuthology.orchestra.run.smithi137.stderr: 80850K .......... .......... .......... .......... .......... 87% 93.7M 0s 2024-03-04T18:06:46.195 INFO:teuthology.orchestra.run.smithi137.stderr: 80900K .......... .......... .......... .......... .......... 87% 82.2M 0s 2024-03-04T18:06:46.195 INFO:teuthology.orchestra.run.smithi137.stderr: 80950K .......... .......... .......... .......... .......... 87% 94.0M 0s 2024-03-04T18:06:46.195 INFO:teuthology.orchestra.run.smithi137.stderr: 81000K .......... .......... .......... .......... .......... 87% 111M 0s 2024-03-04T18:06:46.195 INFO:teuthology.orchestra.run.smithi137.stderr: 81050K .......... .......... .......... .......... .......... 87% 100M 0s 2024-03-04T18:06:46.195 INFO:teuthology.orchestra.run.smithi137.stderr: 81100K .......... .......... .......... .......... .......... 87% 116M 0s 2024-03-04T18:06:46.195 INFO:teuthology.orchestra.run.smithi137.stderr: 81150K .......... .......... .......... .......... .......... 87% 45.1M 0s 2024-03-04T18:06:46.195 INFO:teuthology.orchestra.run.smithi137.stderr: 81200K .......... .......... .......... .......... .......... 87% 150M 0s 2024-03-04T18:06:46.195 INFO:teuthology.orchestra.run.smithi137.stderr: 81250K .......... .......... .......... .......... .......... 87% 90.0M 0s 2024-03-04T18:06:46.195 INFO:teuthology.orchestra.run.smithi137.stderr: 81300K .......... .......... .......... .......... .......... 87% 106M 0s 2024-03-04T18:06:46.195 INFO:teuthology.orchestra.run.smithi137.stderr: 81350K .......... .......... .......... .......... .......... 87% 108M 0s 2024-03-04T18:06:46.195 INFO:teuthology.orchestra.run.smithi137.stderr: 81400K .......... .......... .......... .......... .......... 87% 113M 0s 2024-03-04T18:06:46.196 INFO:teuthology.orchestra.run.smithi137.stderr: 81450K .......... .......... .......... .......... .......... 87% 99.4M 0s 2024-03-04T18:06:46.196 INFO:teuthology.orchestra.run.smithi137.stderr: 81500K .......... .......... .......... .......... .......... 87% 114M 0s 2024-03-04T18:06:46.196 INFO:teuthology.orchestra.run.smithi137.stderr: 81550K .......... .......... .......... .......... .......... 87% 99.2M 0s 2024-03-04T18:06:46.196 INFO:teuthology.orchestra.run.smithi137.stderr: 81600K .......... .......... .......... .......... .......... 87% 122M 0s 2024-03-04T18:06:46.196 INFO:teuthology.orchestra.run.smithi137.stderr: 81650K .......... .......... .......... .......... .......... 87% 63.4M 0s 2024-03-04T18:06:46.196 INFO:teuthology.orchestra.run.smithi137.stderr: 81700K .......... .......... .......... .......... .......... 87% 113M 0s 2024-03-04T18:06:46.196 INFO:teuthology.orchestra.run.smithi137.stderr: 81750K .......... .......... .......... .......... .......... 88% 115M 0s 2024-03-04T18:06:46.196 INFO:teuthology.orchestra.run.smithi137.stderr: 81800K .......... .......... .......... .......... .......... 88% 112M 0s 2024-03-04T18:06:46.196 INFO:teuthology.orchestra.run.smithi137.stderr: 81850K .......... .......... .......... .......... .......... 88% 101M 0s 2024-03-04T18:06:46.196 INFO:teuthology.orchestra.run.smithi137.stderr: 81900K .......... .......... .......... .......... .......... 88% 113M 0s 2024-03-04T18:06:46.196 INFO:teuthology.orchestra.run.smithi137.stderr: 81950K .......... .......... .......... .......... .......... 88% 115M 0s 2024-03-04T18:06:46.196 INFO:teuthology.orchestra.run.smithi137.stderr: 82000K .......... .......... .......... .......... .......... 88% 115M 0s 2024-03-04T18:06:46.196 INFO:teuthology.orchestra.run.smithi137.stderr: 82050K .......... .......... .......... .......... .......... 88% 89.2M 0s 2024-03-04T18:06:46.196 INFO:teuthology.orchestra.run.smithi137.stderr: 82100K .......... .......... .......... .......... .......... 88% 120M 0s 2024-03-04T18:06:46.197 INFO:teuthology.orchestra.run.smithi137.stderr: 82150K .......... .......... .......... .......... .......... 88% 112M 0s 2024-03-04T18:06:46.197 INFO:teuthology.orchestra.run.smithi137.stderr: 82200K .......... .......... .......... .......... .......... 88% 121M 0s 2024-03-04T18:06:46.197 INFO:teuthology.orchestra.run.smithi137.stderr: 82250K .......... .......... .......... .......... .......... 88% 99.0M 0s 2024-03-04T18:06:46.197 INFO:teuthology.orchestra.run.smithi137.stderr: 82300K .......... .......... .......... .......... .......... 88% 117M 0s 2024-03-04T18:06:46.197 INFO:teuthology.orchestra.run.smithi137.stderr: 82350K .......... .......... .......... .......... .......... 88% 112M 0s 2024-03-04T18:06:46.197 INFO:teuthology.orchestra.run.smithi137.stderr: 82400K .......... .......... .......... .......... .......... 88% 117M 0s 2024-03-04T18:06:46.197 INFO:teuthology.orchestra.run.smithi137.stderr: 82450K .......... .......... .......... .......... .......... 88% 94.7M 0s 2024-03-04T18:06:46.197 INFO:teuthology.orchestra.run.smithi137.stderr: 82500K .......... .......... .......... .......... .......... 88% 114M 0s 2024-03-04T18:06:46.197 INFO:teuthology.orchestra.run.smithi137.stderr: 82550K .......... .......... .......... .......... .......... 88% 109M 0s 2024-03-04T18:06:46.197 INFO:teuthology.orchestra.run.smithi137.stderr: 82600K .......... .......... .......... .......... .......... 88% 116M 0s 2024-03-04T18:06:46.197 INFO:teuthology.orchestra.run.smithi137.stderr: 82650K .......... .......... .......... .......... .......... 89% 106M 0s 2024-03-04T18:06:46.197 INFO:teuthology.orchestra.run.smithi137.stderr: 82700K .......... .......... .......... .......... .......... 89% 116M 0s 2024-03-04T18:06:46.197 INFO:teuthology.orchestra.run.smithi137.stderr: 82750K .......... .......... .......... .......... .......... 89% 119M 0s 2024-03-04T18:06:46.197 INFO:teuthology.orchestra.run.smithi137.stderr: 82800K .......... .......... .......... .......... .......... 89% 109M 0s 2024-03-04T18:06:46.198 INFO:teuthology.orchestra.run.smithi137.stderr: 82850K .......... .......... .......... .......... .......... 89% 92.9M 0s 2024-03-04T18:06:46.198 INFO:teuthology.orchestra.run.smithi137.stderr: 82900K .......... .......... .......... .......... .......... 89% 112M 0s 2024-03-04T18:06:46.198 INFO:teuthology.orchestra.run.smithi137.stderr: 82950K .......... .......... .......... .......... .......... 89% 116M 0s 2024-03-04T18:06:46.198 INFO:teuthology.orchestra.run.smithi137.stderr: 83000K .......... .......... .......... .......... .......... 89% 117M 0s 2024-03-04T18:06:46.198 INFO:teuthology.orchestra.run.smithi137.stderr: 83050K .......... .......... .......... .......... .......... 89% 99.7M 0s 2024-03-04T18:06:46.198 INFO:teuthology.orchestra.run.smithi137.stderr: 83100K .......... .......... .......... .......... .......... 89% 114M 0s 2024-03-04T18:06:46.198 INFO:teuthology.orchestra.run.smithi137.stderr: 83150K .......... .......... .......... .......... .......... 89% 117M 0s 2024-03-04T18:06:46.198 INFO:teuthology.orchestra.run.smithi137.stderr: 83200K .......... .......... .......... .......... .......... 89% 114M 0s 2024-03-04T18:06:46.198 INFO:teuthology.orchestra.run.smithi137.stderr: 83250K .......... .......... .......... .......... .......... 89% 91.0M 0s 2024-03-04T18:06:46.198 INFO:teuthology.orchestra.run.smithi137.stderr: 83300K .......... .......... .......... .......... .......... 89% 115M 0s 2024-03-04T18:06:46.198 INFO:teuthology.orchestra.run.smithi137.stderr: 83350K .......... .......... .......... .......... .......... 89% 119M 0s 2024-03-04T18:06:46.198 INFO:teuthology.orchestra.run.smithi137.stderr: 83400K .......... .......... .......... .......... .......... 89% 79.3M 0s 2024-03-04T18:06:46.198 INFO:teuthology.orchestra.run.smithi137.stderr: 83450K .......... .......... .......... .......... .......... 89% 60.2M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 83500K .......... .......... .......... .......... .......... 89% 56.4M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 83550K .......... .......... .......... .......... .......... 89% 116M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 83600K .......... .......... .......... .......... .......... 90% 112M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 83650K .......... .......... .......... .......... .......... 90% 48.2M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 83700K .......... .......... .......... .......... .......... 90% 117M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 83750K .......... .......... .......... .......... .......... 90% 106M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 83800K .......... .......... .......... .......... .......... 90% 121M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 83850K .......... .......... .......... .......... .......... 90% 99.4M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 83900K .......... .......... .......... .......... .......... 90% 105M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 83950K .......... .......... .......... .......... .......... 90% 41.5M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 84000K .......... .......... .......... .......... .......... 90% 112M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 84050K .......... .......... .......... .......... .......... 90% 90.4M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 84100K .......... .......... .......... .......... .......... 90% 114M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 84150K .......... .......... .......... .......... .......... 90% 115M 0s 2024-03-04T18:06:46.199 INFO:teuthology.orchestra.run.smithi137.stderr: 84200K .......... .......... .......... .......... .......... 90% 110M 0s 2024-03-04T18:06:46.200 INFO:teuthology.orchestra.run.smithi137.stderr: 84250K .......... .......... .......... .......... .......... 90% 107M 0s 2024-03-04T18:06:46.200 INFO:teuthology.orchestra.run.smithi137.stderr: 84300K .......... .......... .......... .......... .......... 90% 89.5M 0s 2024-03-04T18:06:46.200 INFO:teuthology.orchestra.run.smithi137.stderr: 84350K .......... .......... .......... .......... .......... 90% 30.8M 0s 2024-03-04T18:06:46.200 INFO:teuthology.orchestra.run.smithi137.stderr: 84400K .......... .......... .......... .......... .......... 90% 119M 0s 2024-03-04T18:06:46.200 INFO:teuthology.orchestra.run.smithi137.stderr: 84450K .......... .......... .......... .......... .......... 90% 91.9M 0s 2024-03-04T18:06:46.200 INFO:teuthology.orchestra.run.smithi137.stderr: 84500K .......... .......... .......... .......... .......... 91% 116M 0s 2024-03-04T18:06:46.200 INFO:teuthology.orchestra.run.smithi137.stderr: 84550K .......... .......... .......... .......... .......... 91% 114M 0s 2024-03-04T18:06:46.200 INFO:teuthology.orchestra.run.smithi137.stderr: 84600K .......... .......... .......... .......... .......... 91% 114M 0s 2024-03-04T18:06:46.200 INFO:teuthology.orchestra.run.smithi137.stderr: 84650K .......... .......... .......... .......... .......... 91% 98.9M 0s 2024-03-04T18:06:46.200 INFO:teuthology.orchestra.run.smithi137.stderr: 84700K .......... .......... .......... .......... .......... 91% 118M 0s 2024-03-04T18:06:46.200 INFO:teuthology.orchestra.run.smithi137.stderr: 84750K .......... .......... .......... .......... .......... 91% 107M 0s 2024-03-04T18:06:46.200 INFO:teuthology.orchestra.run.smithi137.stderr: 84800K .......... .......... .......... .......... .......... 91% 123M 0s 2024-03-04T18:06:46.200 INFO:teuthology.orchestra.run.smithi137.stderr: 84850K .......... .......... .......... .......... .......... 91% 89.9M 0s 2024-03-04T18:06:46.201 INFO:teuthology.orchestra.run.smithi137.stderr: 84900K .......... .......... .......... .......... .......... 91% 112M 0s 2024-03-04T18:06:46.201 INFO:teuthology.orchestra.run.smithi137.stderr: 84950K .......... .......... .......... .......... .......... 91% 114M 0s 2024-03-04T18:06:46.201 INFO:teuthology.orchestra.run.smithi137.stderr: 85000K .......... .......... .......... .......... .......... 91% 111M 0s 2024-03-04T18:06:46.201 INFO:teuthology.orchestra.run.smithi137.stderr: 85050K .......... .......... .......... .......... .......... 91% 105M 0s 2024-03-04T18:06:46.201 INFO:teuthology.orchestra.run.smithi137.stderr: 85100K .......... .......... .......... .......... .......... 91% 105M 0s 2024-03-04T18:06:46.201 INFO:teuthology.orchestra.run.smithi137.stderr: 85150K .......... .......... .......... .......... .......... 91% 120M 0s 2024-03-04T18:06:46.201 INFO:teuthology.orchestra.run.smithi137.stderr: 85200K .......... .......... .......... .......... .......... 91% 94.7M 0s 2024-03-04T18:06:46.201 INFO:teuthology.orchestra.run.smithi137.stderr: 85250K .......... .......... .......... .......... .......... 91% 92.3M 0s 2024-03-04T18:06:46.201 INFO:teuthology.orchestra.run.smithi137.stderr: 85300K .......... .......... .......... .......... .......... 91% 112M 0s 2024-03-04T18:06:46.201 INFO:teuthology.orchestra.run.smithi137.stderr: 85350K .......... .......... .......... .......... .......... 91% 72.8M 0s 2024-03-04T18:06:46.201 INFO:teuthology.orchestra.run.smithi137.stderr: 85400K .......... .......... .......... .......... .......... 91% 122M 0s 2024-03-04T18:06:46.202 INFO:teuthology.orchestra.run.smithi137.stderr: 85450K .......... .......... .......... .......... .......... 92% 101M 0s 2024-03-04T18:06:46.202 INFO:teuthology.orchestra.run.smithi137.stderr: 85500K .......... .......... .......... .......... .......... 92% 112M 0s 2024-03-04T18:06:46.202 INFO:teuthology.orchestra.run.smithi137.stderr: 85550K .......... .......... .......... .......... .......... 92% 113M 0s 2024-03-04T18:06:46.202 INFO:teuthology.orchestra.run.smithi137.stderr: 85600K .......... .......... .......... .......... .......... 92% 119M 0s 2024-03-04T18:06:46.202 INFO:teuthology.orchestra.run.smithi137.stderr: 85650K .......... .......... .......... .......... .......... 92% 80.8M 0s 2024-03-04T18:06:46.202 INFO:teuthology.orchestra.run.smithi137.stderr: 85700K .......... .......... .......... .......... .......... 92% 107M 0s 2024-03-04T18:06:46.202 INFO:teuthology.orchestra.run.smithi137.stderr: 85750K .......... .......... .......... .......... .......... 92% 111M 0s 2024-03-04T18:06:46.202 INFO:teuthology.orchestra.run.smithi137.stderr: 85800K .......... .......... .......... .......... .......... 92% 111M 0s 2024-03-04T18:06:46.202 INFO:teuthology.orchestra.run.smithi137.stderr: 85850K .......... .......... .......... .......... .......... 92% 117M 0s 2024-03-04T18:06:46.202 INFO:teuthology.orchestra.run.smithi137.stderr: 85900K .......... .......... .......... .......... .......... 92% 103M 0s 2024-03-04T18:06:46.202 INFO:teuthology.orchestra.run.smithi137.stderr: 85950K .......... .......... .......... .......... .......... 92% 119M 0s 2024-03-04T18:06:46.202 INFO:teuthology.orchestra.run.smithi137.stderr: 86000K .......... .......... .......... .......... .......... 92% 108M 0s 2024-03-04T18:06:46.202 INFO:teuthology.orchestra.run.smithi137.stderr: 86050K .......... .......... .......... .......... .......... 92% 56.5M 0s 2024-03-04T18:06:46.203 INFO:teuthology.orchestra.run.smithi137.stderr: 86100K .......... .......... .......... .......... .......... 92% 127M 0s 2024-03-04T18:06:46.203 INFO:teuthology.orchestra.run.smithi137.stderr: 86150K .......... .......... .......... .......... .......... 92% 45.6M 0s 2024-03-04T18:06:46.203 INFO:teuthology.orchestra.run.smithi137.stderr: 86200K .......... .......... .......... .......... .......... 92% 36.1M 0s 2024-03-04T18:06:46.203 INFO:teuthology.orchestra.run.smithi137.stderr: 86250K .......... .......... .......... .......... .......... 92% 110M 0s 2024-03-04T18:06:46.203 INFO:teuthology.orchestra.run.smithi137.stderr: 86300K .......... .......... .......... .......... .......... 92% 113M 0s 2024-03-04T18:06:46.203 INFO:teuthology.orchestra.run.smithi137.stderr: 86350K .......... .......... .......... .......... .......... 93% 113M 0s 2024-03-04T18:06:46.203 INFO:teuthology.orchestra.run.smithi137.stderr: 86400K .......... .......... .......... .......... .......... 93% 41.5M 0s 2024-03-04T18:06:46.203 INFO:teuthology.orchestra.run.smithi137.stderr: 86450K .......... .......... .......... .......... .......... 93% 64.3M 0s 2024-03-04T18:06:46.203 INFO:teuthology.orchestra.run.smithi137.stderr: 86500K .......... .......... .......... .......... .......... 93% 110M 0s 2024-03-04T18:06:46.203 INFO:teuthology.orchestra.run.smithi137.stderr: 86550K .......... .......... .......... .......... .......... 93% 113M 0s 2024-03-04T18:06:46.203 INFO:teuthology.orchestra.run.smithi137.stderr: 86600K .......... .......... .......... .......... .......... 93% 114M 0s 2024-03-04T18:06:46.204 INFO:teuthology.orchestra.run.smithi137.stderr: 86650K .......... .......... .......... .......... .......... 93% 107M 0s 2024-03-04T18:06:46.204 INFO:teuthology.orchestra.run.smithi137.stderr: 86700K .......... .......... .......... .......... .......... 93% 101M 0s 2024-03-04T18:06:46.204 INFO:teuthology.orchestra.run.smithi137.stderr: 86750K .......... .......... .......... .......... .......... 93% 120M 0s 2024-03-04T18:06:46.204 INFO:teuthology.orchestra.run.smithi137.stderr: 86800K .......... .......... .......... .......... .......... 93% 112M 0s 2024-03-04T18:06:46.204 INFO:teuthology.orchestra.run.smithi137.stderr: 86850K .......... .......... .......... .......... .......... 93% 45.2M 0s 2024-03-04T18:06:46.204 INFO:teuthology.orchestra.run.smithi137.stderr: 86900K .......... .......... .......... .......... .......... 93% 107M 0s 2024-03-04T18:06:46.204 INFO:teuthology.orchestra.run.smithi137.stderr: 86950K .......... .......... .......... .......... .......... 93% 124M 0s 2024-03-04T18:06:46.204 INFO:teuthology.orchestra.run.smithi137.stderr: 87000K .......... .......... .......... .......... .......... 93% 63.5M 0s 2024-03-04T18:06:46.204 INFO:teuthology.orchestra.run.smithi137.stderr: 87050K .......... .......... .......... .......... .......... 93% 100M 0s 2024-03-04T18:06:46.204 INFO:teuthology.orchestra.run.smithi137.stderr: 87100K .......... .......... .......... .......... .......... 93% 56.6M 0s 2024-03-04T18:06:46.204 INFO:teuthology.orchestra.run.smithi137.stderr: 87150K .......... .......... .......... .......... .......... 93% 84.7M 0s 2024-03-04T18:06:46.204 INFO:teuthology.orchestra.run.smithi137.stderr: 87200K .......... .......... .......... .......... .......... 93% 113M 0s 2024-03-04T18:06:46.205 INFO:teuthology.orchestra.run.smithi137.stderr: 87250K .......... .......... .......... .......... .......... 93% 70.5M 0s 2024-03-04T18:06:46.205 INFO:teuthology.orchestra.run.smithi137.stderr: 87300K .......... .......... .......... .......... .......... 94% 178M 0s 2024-03-04T18:06:46.205 INFO:teuthology.orchestra.run.smithi137.stderr: 87350K .......... .......... .......... .......... .......... 94% 115M 0s 2024-03-04T18:06:46.205 INFO:teuthology.orchestra.run.smithi137.stderr: 87400K .......... .......... .......... .......... .......... 94% 105M 0s 2024-03-04T18:06:46.205 INFO:teuthology.orchestra.run.smithi137.stderr: 87450K .......... .......... .......... .......... .......... 94% 108M 0s 2024-03-04T18:06:46.205 INFO:teuthology.orchestra.run.smithi137.stderr: 87500K .......... .......... .......... .......... .......... 94% 120M 0s 2024-03-04T18:06:46.205 INFO:teuthology.orchestra.run.smithi137.stderr: 87550K .......... .......... .......... .......... .......... 94% 110M 0s 2024-03-04T18:06:46.205 INFO:teuthology.orchestra.run.smithi137.stderr: 87600K .......... .......... .......... .......... .......... 94% 113M 0s 2024-03-04T18:06:46.205 INFO:teuthology.orchestra.run.smithi137.stderr: 87650K .......... .......... .......... .......... .......... 94% 90.8M 0s 2024-03-04T18:06:46.205 INFO:teuthology.orchestra.run.smithi137.stderr: 87700K .......... .......... .......... .......... .......... 94% 114M 0s 2024-03-04T18:06:46.205 INFO:teuthology.orchestra.run.smithi137.stderr: 87750K .......... .......... .......... .......... .......... 94% 114M 0s 2024-03-04T18:06:46.205 INFO:teuthology.orchestra.run.smithi137.stderr: 87800K .......... .......... .......... .......... .......... 94% 117M 0s 2024-03-04T18:06:46.205 INFO:teuthology.orchestra.run.smithi137.stderr: 87850K .......... .......... .......... .......... .......... 94% 104M 0s 2024-03-04T18:06:46.206 INFO:teuthology.orchestra.run.smithi137.stderr: 87900K .......... .......... .......... .......... .......... 94% 118M 0s 2024-03-04T18:06:46.206 INFO:teuthology.orchestra.run.smithi137.stderr: 87950K .......... .......... .......... .......... .......... 94% 112M 0s 2024-03-04T18:06:46.206 INFO:teuthology.orchestra.run.smithi137.stderr: 88000K .......... .......... .......... .......... .......... 94% 123M 0s 2024-03-04T18:06:46.206 INFO:teuthology.orchestra.run.smithi137.stderr: 88050K .......... .......... .......... .......... .......... 94% 83.3M 0s 2024-03-04T18:06:46.206 INFO:teuthology.orchestra.run.smithi137.stderr: 88100K .......... .......... .......... .......... .......... 94% 106M 0s 2024-03-04T18:06:46.206 INFO:teuthology.orchestra.run.smithi137.stderr: 88150K .......... .......... .......... .......... .......... 94% 113M 0s 2024-03-04T18:06:46.206 INFO:teuthology.orchestra.run.smithi137.stderr: 88200K .......... .......... .......... .......... .......... 94% 121M 0s 2024-03-04T18:06:46.206 INFO:teuthology.orchestra.run.smithi137.stderr: 88250K .......... .......... .......... .......... .......... 95% 114M 0s 2024-03-04T18:06:46.206 INFO:teuthology.orchestra.run.smithi137.stderr: 88300K .......... .......... .......... .......... .......... 95% 103M 0s 2024-03-04T18:06:46.206 INFO:teuthology.orchestra.run.smithi137.stderr: 88350K .......... .......... .......... .......... .......... 95% 120M 0s 2024-03-04T18:06:46.207 INFO:teuthology.orchestra.run.smithi137.stderr: 88400K .......... .......... .......... .......... .......... 95% 114M 0s 2024-03-04T18:06:46.207 INFO:teuthology.orchestra.run.smithi137.stderr: 88450K .......... .......... .......... .......... .......... 95% 90.8M 0s 2024-03-04T18:06:46.207 INFO:teuthology.orchestra.run.smithi137.stderr: 88500K .......... .......... .......... .......... .......... 95% 114M 0s 2024-03-04T18:06:46.207 INFO:teuthology.orchestra.run.smithi137.stderr: 88550K .......... .......... .......... .......... .......... 95% 107M 0s 2024-03-04T18:06:46.207 INFO:teuthology.orchestra.run.smithi137.stderr: 88600K .......... .......... .......... .......... .......... 95% 74.7M 0s 2024-03-04T18:06:46.207 INFO:teuthology.orchestra.run.smithi137.stderr: 88650K .......... .......... .......... .......... .......... 95% 171M 0s 2024-03-04T18:06:46.207 INFO:teuthology.orchestra.run.smithi137.stderr: 88700K .......... .......... .......... .......... .......... 95% 63.6M 0s 2024-03-04T18:06:46.207 INFO:teuthology.orchestra.run.smithi137.stderr: 88750K .......... .......... .......... .......... .......... 95% 78.5M 0s 2024-03-04T18:06:46.207 INFO:teuthology.orchestra.run.smithi137.stderr: 88800K .......... .......... .......... .......... .......... 95% 34.6M 0s 2024-03-04T18:06:46.207 INFO:teuthology.orchestra.run.smithi137.stderr: 88850K .......... .......... .......... .......... .......... 95% 69.8M 0s 2024-03-04T18:06:46.207 INFO:teuthology.orchestra.run.smithi137.stderr: 88900K .......... .......... .......... .......... .......... 95% 34.2M 0s 2024-03-04T18:06:46.207 INFO:teuthology.orchestra.run.smithi137.stderr: 88950K .......... .......... .......... .......... .......... 95% 118M 0s 2024-03-04T18:06:46.207 INFO:teuthology.orchestra.run.smithi137.stderr: 89000K .......... .......... .......... .......... .......... 95% 74.5M 0s 2024-03-04T18:06:46.208 INFO:teuthology.orchestra.run.smithi137.stderr: 89050K .......... .......... .......... .......... .......... 95% 104M 0s 2024-03-04T18:06:46.208 INFO:teuthology.orchestra.run.smithi137.stderr: 89100K .......... .......... .......... .......... .......... 95% 109M 0s 2024-03-04T18:06:46.208 INFO:teuthology.orchestra.run.smithi137.stderr: 89150K .......... .......... .......... .......... .......... 96% 117M 0s 2024-03-04T18:06:46.208 INFO:teuthology.orchestra.run.smithi137.stderr: 89200K .......... .......... .......... .......... .......... 96% 25.7M 0s 2024-03-04T18:06:46.208 INFO:teuthology.orchestra.run.smithi137.stderr: 89250K .......... .......... .......... .......... .......... 96% 48.6M 0s 2024-03-04T18:06:46.208 INFO:teuthology.orchestra.run.smithi137.stderr: 89300K .......... .......... .......... .......... .......... 96% 107M 0s 2024-03-04T18:06:46.208 INFO:teuthology.orchestra.run.smithi137.stderr: 89350K .......... .......... .......... .......... .......... 96% 108M 0s 2024-03-04T18:06:46.208 INFO:teuthology.orchestra.run.smithi137.stderr: 89400K .......... .......... .......... .......... .......... 96% 116M 0s 2024-03-04T18:06:46.208 INFO:teuthology.orchestra.run.smithi137.stderr: 89450K .......... .......... .......... .......... .......... 96% 92.4M 0s 2024-03-04T18:06:46.208 INFO:teuthology.orchestra.run.smithi137.stderr: 89500K .......... .......... .......... .......... .......... 96% 109M 0s 2024-03-04T18:06:46.208 INFO:teuthology.orchestra.run.smithi137.stderr: 89550K .......... .......... .......... .......... .......... 96% 116M 0s 2024-03-04T18:06:46.209 INFO:teuthology.orchestra.run.smithi137.stderr: 89600K .......... .......... .......... .......... .......... 96% 115M 0s 2024-03-04T18:06:46.209 INFO:teuthology.orchestra.run.smithi137.stderr: 89650K .......... .......... .......... .......... .......... 96% 92.1M 0s 2024-03-04T18:06:46.209 INFO:teuthology.orchestra.run.smithi137.stderr: 89700K .......... .......... .......... .......... .......... 96% 112M 0s 2024-03-04T18:06:46.209 INFO:teuthology.orchestra.run.smithi137.stderr: 89750K .......... .......... .......... .......... .......... 96% 46.0M 0s 2024-03-04T18:06:46.209 INFO:teuthology.orchestra.run.smithi137.stderr: 89800K .......... .......... .......... .......... .......... 96% 114M 0s 2024-03-04T18:06:46.209 INFO:teuthology.orchestra.run.smithi137.stderr: 89850K .......... .......... .......... .......... .......... 96% 110M 0s 2024-03-04T18:06:46.209 INFO:teuthology.orchestra.run.smithi137.stderr: 89900K .......... .......... .......... .......... .......... 96% 32.1M 0s 2024-03-04T18:06:46.209 INFO:teuthology.orchestra.run.smithi137.stderr: 89950K .......... .......... .......... .......... .......... 96% 117M 0s 2024-03-04T18:06:46.209 INFO:teuthology.orchestra.run.smithi137.stderr: 90000K .......... .......... .......... .......... .......... 96% 116M 0s 2024-03-04T18:06:46.209 INFO:teuthology.orchestra.run.smithi137.stderr: 90050K .......... .......... .......... .......... .......... 96% 61.9M 0s 2024-03-04T18:06:46.210 INFO:teuthology.orchestra.run.smithi137.stderr: 90100K .......... .......... .......... .......... .......... 97% 121M 0s 2024-03-04T18:06:46.210 INFO:teuthology.orchestra.run.smithi137.stderr: 90150K .......... .......... .......... .......... .......... 97% 113M 0s 2024-03-04T18:06:46.210 INFO:teuthology.orchestra.run.smithi137.stderr: 90200K .......... .......... .......... .......... .......... 97% 109M 0s 2024-03-04T18:06:46.210 INFO:teuthology.orchestra.run.smithi137.stderr: 90250K .......... .......... .......... .......... .......... 97% 103M 0s 2024-03-04T18:06:46.210 INFO:teuthology.orchestra.run.smithi137.stderr: 90300K .......... .......... .......... .......... .......... 97% 119M 0s 2024-03-04T18:06:46.210 INFO:teuthology.orchestra.run.smithi137.stderr: 90350K .......... .......... .......... .......... .......... 97% 78.8M 0s 2024-03-04T18:06:46.210 INFO:teuthology.orchestra.run.smithi137.stderr: 90400K .......... .......... .......... .......... .......... 97% 187M 0s 2024-03-04T18:06:46.210 INFO:teuthology.orchestra.run.smithi137.stderr: 90450K .......... .......... .......... .......... .......... 97% 59.8M 0s 2024-03-04T18:06:46.210 INFO:teuthology.orchestra.run.smithi137.stderr: 90500K .......... .......... .......... .......... .......... 97% 64.6M 0s 2024-03-04T18:06:46.210 INFO:teuthology.orchestra.run.smithi137.stderr: 90550K .......... .......... .......... .......... .......... 97% 125M 0s 2024-03-04T18:06:46.210 INFO:teuthology.orchestra.run.smithi137.stderr: 90600K .......... .......... .......... .......... .......... 97% 109M 0s 2024-03-04T18:06:46.210 INFO:teuthology.orchestra.run.smithi137.stderr: 90650K .......... .......... .......... .......... .......... 97% 107M 0s 2024-03-04T18:06:46.211 INFO:teuthology.orchestra.run.smithi137.stderr: 90700K .......... .......... .......... .......... .......... 97% 108M 0s 2024-03-04T18:06:46.211 INFO:teuthology.orchestra.run.smithi137.stderr: 90750K .......... .......... .......... .......... .......... 97% 115M 0s 2024-03-04T18:06:46.211 INFO:teuthology.orchestra.run.smithi137.stderr: 90800K .......... .......... .......... .......... .......... 97% 113M 0s 2024-03-04T18:06:46.211 INFO:teuthology.orchestra.run.smithi137.stderr: 90850K .......... .......... .......... .......... .......... 97% 87.6M 0s 2024-03-04T18:06:46.211 INFO:teuthology.orchestra.run.smithi137.stderr: 90900K .......... .......... .......... .......... .......... 97% 97.2M 0s 2024-03-04T18:06:46.211 INFO:teuthology.orchestra.run.smithi137.stderr: 90950K .......... .......... .......... .......... .......... 97% 114M 0s 2024-03-04T18:06:46.211 INFO:teuthology.orchestra.run.smithi137.stderr: 91000K .......... .......... .......... .......... .......... 98% 114M 0s 2024-03-04T18:06:46.211 INFO:teuthology.orchestra.run.smithi137.stderr: 91050K .......... .......... .......... .......... .......... 98% 101M 0s 2024-03-04T18:06:46.211 INFO:teuthology.orchestra.run.smithi137.stderr: 91100K .......... .......... .......... .......... .......... 98% 116M 0s 2024-03-04T18:06:46.211 INFO:teuthology.orchestra.run.smithi137.stderr: 91150K .......... .......... .......... .......... .......... 98% 112M 0s 2024-03-04T18:06:46.211 INFO:teuthology.orchestra.run.smithi137.stderr: 91200K .......... .......... .......... .......... .......... 98% 115M 0s 2024-03-04T18:06:46.212 INFO:teuthology.orchestra.run.smithi137.stderr: 91250K .......... .......... .......... .......... .......... 98% 90.8M 0s 2024-03-04T18:06:46.212 INFO:teuthology.orchestra.run.smithi137.stderr: 91300K .......... .......... .......... .......... .......... 98% 109M 0s 2024-03-04T18:06:46.212 INFO:teuthology.orchestra.run.smithi137.stderr: 91350K .......... .......... .......... .......... .......... 98% 44.6M 0s 2024-03-04T18:06:46.212 INFO:teuthology.orchestra.run.smithi137.stderr: 91400K .......... .......... .......... .......... .......... 98% 48.7M 0s 2024-03-04T18:06:46.212 INFO:teuthology.orchestra.run.smithi137.stderr: 91450K .......... .......... .......... .......... .......... 98% 108M 0s 2024-03-04T18:06:46.212 INFO:teuthology.orchestra.run.smithi137.stderr: 91500K .......... .......... .......... .......... .......... 98% 52.6M 0s 2024-03-04T18:06:46.212 INFO:teuthology.orchestra.run.smithi137.stderr: 91550K .......... .......... .......... .......... .......... 98% 25.9M 0s 2024-03-04T18:06:46.212 INFO:teuthology.orchestra.run.smithi137.stderr: 91600K .......... .......... .......... .......... .......... 98% 75.5M 0s 2024-03-04T18:06:46.212 INFO:teuthology.orchestra.run.smithi137.stderr: 91650K .......... .......... .......... .......... .......... 98% 64.0M 0s 2024-03-04T18:06:46.212 INFO:teuthology.orchestra.run.smithi137.stderr: 91700K .......... .......... .......... .......... .......... 98% 35.5M 0s 2024-03-04T18:06:46.212 INFO:teuthology.orchestra.run.smithi137.stderr: 91750K .......... .......... .......... .......... .......... 98% 115M 0s 2024-03-04T18:06:46.212 INFO:teuthology.orchestra.run.smithi137.stderr: 91800K .......... .......... .......... .......... .......... 98% 80.6M 0s 2024-03-04T18:06:46.212 INFO:teuthology.orchestra.run.smithi137.stderr: 91850K .......... .......... .......... .......... .......... 98% 58.9M 0s 2024-03-04T18:06:46.212 INFO:teuthology.orchestra.run.smithi137.stderr: 91900K .......... .......... .......... .......... .......... 98% 78.8M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 91950K .......... .......... .......... .......... .......... 99% 91.3M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 92000K .......... .......... .......... .......... .......... 99% 29.5M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 92050K .......... .......... .......... .......... .......... 99% 21.3M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 92100K .......... .......... .......... .......... .......... 99% 78.0M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 92150K .......... .......... .......... .......... .......... 99% 109M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 92200K .......... .......... .......... .......... .......... 99% 117M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 92250K .......... .......... .......... .......... .......... 99% 96.4M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 92300K .......... .......... .......... .......... .......... 99% 113M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 92350K .......... .......... .......... .......... .......... 99% 107M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 92400K .......... .......... .......... .......... .......... 99% 118M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 92450K .......... .......... .......... .......... .......... 99% 90.9M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 92500K .......... .......... .......... .......... .......... 99% 108M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 92550K .......... .......... .......... .......... .......... 99% 113M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 92600K .......... .......... .......... .......... .......... 99% 28.6M 0s 2024-03-04T18:06:46.213 INFO:teuthology.orchestra.run.smithi137.stderr: 92650K .......... .......... .......... .......... .......... 99% 102M 0s 2024-03-04T18:06:46.214 INFO:teuthology.orchestra.run.smithi137.stderr: 92700K .......... .......... .......... .......... .......... 99% 27.7M 0s 2024-03-04T18:06:46.214 INFO:teuthology.orchestra.run.smithi137.stderr: 92750K .......... .......... .......... .......... .......... 99% 84.0M 0s 2024-03-04T18:06:46.214 INFO:teuthology.orchestra.run.smithi137.stderr: 92800K .......... .......... .......... .......... .......... 99% 55.4M 0s 2024-03-04T18:06:46.214 INFO:teuthology.orchestra.run.smithi137.stderr: 92850K .......... .......... .......... .......... .......... 99% 111M 0s 2024-03-04T18:06:46.214 INFO:teuthology.orchestra.run.smithi137.stderr: 92900K 100% 134G=1.0s 2024-03-04T18:06:46.214 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-03-04T18:06:46.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04 18:06:45 (93.7 MB/s) - 'gradle-6.0.1-bin.zip' saved [95129672/95129672] 2024-03-04T18:06:46.214 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-03-04T18:06:46.240 INFO:teuthology.orchestra.run.smithi137.stdout:Archive: gradle-6.0.1-bin.zip 2024-03-04T18:06:46.240 INFO:teuthology.orchestra.run.smithi137.stdout: creating: /opt/gradle/gradle-6.0.1/ 2024-03-04T18:06:46.240 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/getting-started.html 2024-03-04T18:06:46.241 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/LICENSE 2024-03-04T18:06:46.241 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/NOTICE 2024-03-04T18:06:46.241 INFO:teuthology.orchestra.run.smithi137.stdout: creating: /opt/gradle/gradle-6.0.1/init.d/ 2024-03-04T18:06:46.241 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/init.d/readme.txt 2024-03-04T18:06:46.241 INFO:teuthology.orchestra.run.smithi137.stdout: creating: /opt/gradle/gradle-6.0.1/bin/ 2024-03-04T18:06:46.241 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/bin/gradle 2024-03-04T18:06:46.241 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/bin/gradle.bat 2024-03-04T18:06:46.241 INFO:teuthology.orchestra.run.smithi137.stdout: creating: /opt/gradle/gradle-6.0.1/lib/ 2024-03-04T18:06:46.241 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-api-metadata-6.0.1.jar 2024-03-04T18:06:46.241 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-launcher-6.0.1.jar 2024-03-04T18:06:46.242 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-runtime-api-info-6.0.1.jar 2024-03-04T18:06:46.242 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-kotlin-dsl-6.0.1.jar 2024-03-04T18:06:46.242 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-tooling-api-6.0.1.jar 2024-03-04T18:06:46.242 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-wrapper-6.0.1.jar 2024-03-04T18:06:46.242 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-installation-beacon-6.0.1.jar 2024-03-04T18:06:46.242 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-bootstrap-6.0.1.jar 2024-03-04T18:06:46.242 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-core-6.0.1.jar 2024-03-04T18:06:46.242 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-file-collections-6.0.1.jar 2024-03-04T18:06:46.242 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-model-groovy-6.0.1.jar 2024-03-04T18:06:46.242 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-execution-6.0.1.jar 2024-03-04T18:06:46.242 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-model-core-6.0.1.jar 2024-03-04T18:06:46.243 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-cache-packaging-6.0.1.jar 2024-03-04T18:06:46.243 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-cache-6.0.1.jar 2024-03-04T18:06:46.243 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-core-api-6.0.1.jar 2024-03-04T18:06:46.243 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-persistent-cache-6.0.1.jar 2024-03-04T18:06:46.243 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-worker-processes-6.0.1.jar 2024-03-04T18:06:46.243 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-logging-6.0.1.jar 2024-03-04T18:06:46.243 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-jvm-services-6.0.1.jar 2024-03-04T18:06:46.243 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-process-services-6.0.1.jar 2024-03-04T18:06:46.243 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-resources-6.0.1.jar 2024-03-04T18:06:46.243 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-messaging-6.0.1.jar 2024-03-04T18:06:46.243 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-native-6.0.1.jar 2024-03-04T18:06:46.243 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-base-services-groovy-6.0.1.jar 2024-03-04T18:06:46.244 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-base-services-6.0.1.jar 2024-03-04T18:06:46.244 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-option-6.0.1.jar 2024-03-04T18:06:46.244 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-cli-6.0.1.jar 2024-03-04T18:06:46.244 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-snapshots-6.0.1.jar 2024-03-04T18:06:46.244 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-files-6.0.1.jar 2024-03-04T18:06:46.244 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-kotlin-dsl-tooling-models-6.0.1.jar 2024-03-04T18:06:46.244 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-compiler-embeddable-1.3.50-patched-for-gradle-6.0.1.jar 2024-03-04T18:06:46.244 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-hashing-6.0.1.jar 2024-03-04T18:06:46.244 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-pineapple-6.0.1.jar 2024-03-04T18:06:46.244 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-docs-6.0.1.jar 2024-03-04T18:06:46.244 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/groovy-all-1.3-2.5.8.jar 2024-03-04T18:06:46.245 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jul-to-slf4j-1.7.28.jar 2024-03-04T18:06:46.245 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/log4j-over-slf4j-1.7.28.jar 2024-03-04T18:06:46.245 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jcl-over-slf4j-1.7.28.jar 2024-03-04T18:06:46.245 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/slf4j-api-1.7.28.jar 2024-03-04T18:06:46.245 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/guava-27.1-android.jar 2024-03-04T18:06:46.245 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-io-2.6.jar 2024-03-04T18:06:46.245 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-lang-2.6.jar 2024-03-04T18:06:46.245 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-commons-7.1.jar 2024-03-04T18:06:46.245 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-analysis-7.1.jar 2024-03-04T18:06:46.246 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-tree-7.1.jar 2024-03-04T18:06:46.246 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-7.1.jar 2024-03-04T18:06:46.246 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/ant-1.10.7.jar 2024-03-04T18:06:46.246 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-jdk8-1.3.50.jar 2024-03-04T18:06:46.246 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/javax.inject-1.jar 2024-03-04T18:06:46.246 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-common-1.3.50.jar 2024-03-04T18:06:46.246 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-jvm-1.3.50.jar 2024-03-04T18:06:46.246 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-jvm-host-embeddable-1.3.50.jar 2024-03-04T18:06:46.246 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-compiler-embeddable-1.3.50.jar 2024-03-04T18:06:46.246 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-compiler-impl-embeddable-1.3.50.jar 2024-03-04T18:06:46.246 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-sam-with-receiver-compiler-plugin-1.3.50.jar 2024-03-04T18:06:46.246 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlinx-metadata-jvm-0.1.0.jar 2024-03-04T18:06:46.247 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-compress-1.19.jar 2024-03-04T18:06:46.247 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/fastutil-8.3.0-min.jar 2024-03-04T18:06:46.247 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jansi-1.18.jar 2024-03-04T18:06:46.247 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jsr305-3.0.2.jar 2024-03-04T18:06:46.247 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kryo-2.24.0.jar 2024-03-04T18:06:46.247 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-0.18.jar 2024-03-04T18:06:46.247 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/objenesis-2.6.jar 2024-03-04T18:06:46.247 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/trove4j-1.0.20181211.jar 2024-03-04T18:06:46.247 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/xml-apis-1.4.01.jar 2024-03-04T18:06:46.247 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-reflect-1.3.50.jar 2024-03-04T18:06:46.247 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-jdk7-1.3.50.jar 2024-03-04T18:06:46.248 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-1.3.50.jar 2024-03-04T18:06:46.248 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/annotations-13.0.jar 2024-03-04T18:06:46.248 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/ant-launcher-1.10.7.jar 2024-03-04T18:06:46.248 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/minlog-1.2.jar 2024-03-04T18:06:46.248 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-script-runtime-1.3.50.jar 2024-03-04T18:06:46.248 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-daemon-embeddable-1.3.50.jar 2024-03-04T18:06:46.248 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/failureaccess-1.0.1.jar 2024-03-04T18:06:46.248 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-common-1.3.50.jar 2024-03-04T18:06:46.248 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-amd64-libcpp-0.18.jar 2024-03-04T18:06:46.248 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-amd64-libstdcpp-0.18.jar 2024-03-04T18:06:46.249 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-i386-libcpp-0.18.jar 2024-03-04T18:06:46.249 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-i386-libstdcpp-0.18.jar 2024-03-04T18:06:46.249 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-aarch64-0.18.jar 2024-03-04T18:06:46.249 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-0.18.jar 2024-03-04T18:06:46.249 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-0.18.jar 2024-03-04T18:06:46.249 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-osx-amd64-0.18.jar 2024-03-04T18:06:46.249 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-amd64-0.18.jar 2024-03-04T18:06:46.249 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-amd64-min-0.18.jar 2024-03-04T18:06:46.249 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-i386-0.18.jar 2024-03-04T18:06:46.249 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-i386-min-0.18.jar 2024-03-04T18:06:46.249 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-aarch64-ncurses5-0.18.jar 2024-03-04T18:06:46.250 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-ncurses5-0.18.jar 2024-03-04T18:06:46.250 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-ncurses6-0.18.jar 2024-03-04T18:06:46.250 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-ncurses5-0.18.jar 2024-03-04T18:06:46.250 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-ncurses6-0.18.jar 2024-03-04T18:06:46.250 INFO:teuthology.orchestra.run.smithi137.stdout: creating: /opt/gradle/gradle-6.0.1/lib/plugins/ 2024-03-04T18:06:46.250 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-antlr-6.0.1.jar 2024-03-04T18:06:46.250 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-code-quality-6.0.1.jar 2024-03-04T18:06:46.250 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-signing-6.0.1.jar 2024-03-04T18:06:46.250 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-tooling-native-6.0.1.jar 2024-03-04T18:06:46.250 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-native-6.0.1.jar 2024-03-04T18:06:46.250 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-native-6.0.1.jar 2024-03-04T18:06:46.250 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-native-6.0.1.jar 2024-03-04T18:06:46.251 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-kotlin-dsl-provider-plugins-6.0.1.jar 2024-03-04T18:06:46.251 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugin-development-6.0.1.jar 2024-03-04T18:06:46.251 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-maven-6.0.1.jar 2024-03-04T18:06:46.251 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-play-6.0.1.jar 2024-03-04T18:06:46.251 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-6.0.1.jar 2024-03-04T18:06:46.251 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-scala-6.0.1.jar 2024-03-04T18:06:46.251 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ear-6.0.1.jar 2024-03-04T18:06:46.251 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-play-6.0.1.jar 2024-03-04T18:06:46.251 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-javascript-6.0.1.jar 2024-03-04T18:06:46.251 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ivy-6.0.1.jar 2024-03-04T18:06:46.251 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-jacoco-6.0.1.jar 2024-03-04T18:06:46.252 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-init-6.0.1.jar 2024-03-04T18:06:46.252 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-kotlin-dsl-tooling-builders-6.0.1.jar 2024-03-04T18:06:46.252 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugins-6.0.1.jar 2024-03-04T18:06:46.252 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-test-kit-6.0.1.jar 2024-03-04T18:06:46.252 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-groovy-6.0.1.jar 2024-03-04T18:06:46.252 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-scala-6.0.1.jar 2024-03-04T18:06:46.252 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-junit-platform-6.0.1.jar 2024-03-04T18:06:46.252 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-tooling-api-builders-6.0.1.jar 2024-03-04T18:06:46.252 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-jvm-6.0.1.jar 2024-03-04T18:06:46.252 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-java-6.0.1.jar 2024-03-04T18:06:46.253 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-jvm-6.0.1.jar 2024-03-04T18:06:46.253 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-jvm-6.0.1.jar 2024-03-04T18:06:46.253 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-native-6.0.1.jar 2024-03-04T18:06:46.253 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-diagnostics-6.0.1.jar 2024-03-04T18:06:46.253 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-publish-6.0.1.jar 2024-03-04T18:06:46.253 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-base-6.0.1.jar 2024-03-04T18:06:46.253 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-base-6.0.1.jar 2024-03-04T18:06:46.253 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugin-use-6.0.1.jar 2024-03-04T18:06:46.253 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-instant-execution-6.0.1.jar 2024-03-04T18:06:46.253 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-version-control-6.0.1.jar 2024-03-04T18:06:46.253 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-dependency-management-6.0.1.jar 2024-03-04T18:06:46.254 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-composite-builds-6.0.1.jar 2024-03-04T18:06:46.254 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-reporting-6.0.1.jar 2024-03-04T18:06:46.254 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-s3-6.0.1.jar 2024-03-04T18:06:46.254 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-gcs-6.0.1.jar 2024-03-04T18:06:46.254 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-cache-http-6.0.1.jar 2024-03-04T18:06:46.254 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-http-6.0.1.jar 2024-03-04T18:06:46.254 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-sftp-6.0.1.jar 2024-03-04T18:06:46.254 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-profile-6.0.1.jar 2024-03-04T18:06:46.254 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-workers-6.0.1.jar 2024-03-04T18:06:46.254 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/org.eclipse.jgit-5.5.0.201909110433-r.jar 2024-03-04T18:06:46.254 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-http-3.0.0.jar 2024-03-04T18:06:46.255 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-http-shared-3.0.0.jar 2024-03-04T18:06:46.255 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-api-services-storage-v1-rev136-1.25.0.jar 2024-03-04T18:06:46.255 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-api-client-1.25.0.jar 2024-03-04T18:06:46.255 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/asm-util-7.1.jar 2024-03-04T18:06:46.255 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/ivy-2.3.0.jar 2024-03-04T18:06:46.255 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-compat-3.0.4.jar 2024-03-04T18:06:46.255 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-core-3.0.4.jar 2024-03-04T18:06:46.255 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/pmaven-common-0.8-20100325.jar 2024-03-04T18:06:46.255 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/pmaven-groovy-0.8-20100325.jar 2024-03-04T18:06:46.255 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-file-3.0.0.jar 2024-03-04T18:06:46.255 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-container-default-1.7.1.jar 2024-03-04T18:06:46.256 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-connector-wagon-1.13.1.jar 2024-03-04T18:06:46.256 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcpg-jdk15on-1.63.jar 2024-03-04T18:06:46.256 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/rhino-1.7.10.jar 2024-03-04T18:06:46.256 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gson-2.8.5.jar 2024-03-04T18:06:46.256 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/simple-4.1.21.jar 2024-03-04T18:06:46.256 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jatl-0.2.3.jar 2024-03-04T18:06:46.256 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-plugin-api-3.0.4.jar 2024-03-04T18:06:46.256 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-oauth-client-1.25.0.jar 2024-03-04T18:06:46.256 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-http-client-jackson2-1.25.0.jar 2024-03-04T18:06:46.256 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-http-client-1.25.0.jar 2024-03-04T18:06:46.257 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/testng-6.3.1.jar 2024-03-04T18:06:46.257 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/snakeyaml-1.17.jar 2024-03-04T18:06:46.257 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/dd-plist-1.21.jar 2024-03-04T18:06:46.257 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-4.12.jar 2024-03-04T18:06:46.257 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bsh-2.0b6.jar 2024-03-04T18:06:46.257 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-launcher-1.3.1.jar 2024-03-04T18:06:46.257 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-s3-1.11.633.jar 2024-03-04T18:06:46.257 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-kms-1.11.633.jar 2024-03-04T18:06:46.257 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-core-1.11.633.jar 2024-03-04T18:06:46.257 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/httpclient-4.5.10.jar 2024-03-04T18:06:46.257 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jcifs-1.3.17.jar 2024-03-04T18:06:46.257 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/nekohtml-1.9.22.jar 2024-03-04T18:06:46.258 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/xercesImpl-2.12.0.jar 2024-03-04T18:06:46.258 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jsch-0.1.55.jar 2024-03-04T18:06:46.258 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jaxb-impl-2.3.2.jar 2024-03-04T18:06:46.258 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jmespath-java-1.11.633.jar 2024-03-04T18:06:46.258 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-databind-2.9.10.jar 2024-03-04T18:06:46.258 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-core-2.9.10.jar 2024-03-04T18:06:46.258 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-annotations-2.9.10.jar 2024-03-04T18:06:46.258 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/joda-time-2.10.4.jar 2024-03-04T18:06:46.259 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcpkix-jdk15on-1.61.jar 2024-03-04T18:06:46.273 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcprov-jdk15on-1.63.jar 2024-03-04T18:06:46.276 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/commons-codec-1.13.jar 2024-03-04T18:06:46.276 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-aether-provider-3.0.4.jar 2024-03-04T18:06:46.277 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-impl-1.13.1.jar 2024-03-04T18:06:46.278 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-util-1.13.1.jar 2024-03-04T18:06:46.278 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-spi-1.13.1.jar 2024-03-04T18:06:46.279 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-api-1.13.1.jar 2024-03-04T18:06:46.279 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/hamcrest-core-1.3.jar 2024-03-04T18:06:46.281 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/httpcore-4.4.12.jar 2024-03-04T18:06:46.282 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jcommander-1.72.jar 2024-03-04T18:06:46.282 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jzlib-1.1.3.jar 2024-03-04T18:06:46.282 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-artifact-3.0.4.jar 2024-03-04T18:06:46.283 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-model-builder-3.0.4.jar 2024-03-04T18:06:46.286 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-model-3.0.4.jar 2024-03-04T18:06:46.286 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-repository-metadata-3.0.4.jar 2024-03-04T18:06:46.286 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-settings-builder-3.0.4.jar 2024-03-04T18:06:46.286 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-settings-3.0.4.jar 2024-03-04T18:06:46.286 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-sec-dispatcher-1.3.jar 2024-03-04T18:06:46.286 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-cipher-1.7.jar 2024-03-04T18:06:46.287 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-classworlds-2.5.1.jar 2024-03-04T18:06:46.287 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-component-annotations-1.5.5.jar 2024-03-04T18:06:46.288 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-interpolation-1.14.jar 2024-03-04T18:06:46.288 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-provider-api-3.0.0.jar 2024-03-04T18:06:46.288 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-utils-3.1.0.jar 2024-03-04T18:06:46.289 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/xbean-reflect-3.7.jar 2024-03-04T18:06:46.293 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/ion-java-1.0.2.jar 2024-03-04T18:06:46.294 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-engine-1.3.1.jar 2024-03-04T18:06:46.295 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-commons-1.3.1.jar 2024-03-04T18:06:46.295 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/apiguardian-api-1.0.0.jar 2024-03-04T18:06:46.295 INFO:teuthology.orchestra.run.smithi137.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/opentest4j-1.1.1.jar 2024-03-04T18:06:46.326 INFO:teuthology.orchestra.run.smithi137.stdout:export PATH=/opt/gradle/gradle-6.0.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin 2024-03-04T18:06:46.743 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:06:46.743 INFO:teuthology.orchestra.run.smithi137.stdout:Welcome to Gradle 6.0.1! 2024-03-04T18:06:46.743 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:06:46.743 INFO:teuthology.orchestra.run.smithi137.stdout:Here are the highlights of this release: 2024-03-04T18:06:46.743 INFO:teuthology.orchestra.run.smithi137.stdout: - Substantial improvements in dependency management, including 2024-03-04T18:06:46.743 INFO:teuthology.orchestra.run.smithi137.stdout: - Publishing Gradle Module Metadata in addition to pom.xml 2024-03-04T18:06:46.743 INFO:teuthology.orchestra.run.smithi137.stdout: - Advanced control of transitive versions 2024-03-04T18:06:46.743 INFO:teuthology.orchestra.run.smithi137.stdout: - Support for optional features and dependencies 2024-03-04T18:06:46.743 INFO:teuthology.orchestra.run.smithi137.stdout: - Rules to tweak published metadata 2024-03-04T18:06:46.743 INFO:teuthology.orchestra.run.smithi137.stdout: - Support for Java 13 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout: - Faster incremental Java and Groovy compilation 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout: - New Zinc compiler for Scala 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout: - VS2019 support 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout: - Support for Gradle Enterprise plugin 3.0 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout:For more details see https://docs.gradle.org/6.0.1/release-notes.html 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout:------------------------------------------------------------ 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout:Gradle 6.0.1 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout:------------------------------------------------------------ 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout:Build time: 2019-11-18 20:25:01 UTC 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout:Revision: fad121066a68c4701acd362daf4287a7c309a0f5 2024-03-04T18:06:46.744 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:06:46.745 INFO:teuthology.orchestra.run.smithi137.stdout:Kotlin: 1.3.50 2024-03-04T18:06:46.745 INFO:teuthology.orchestra.run.smithi137.stdout:Groovy: 2.5.8 2024-03-04T18:06:46.745 INFO:teuthology.orchestra.run.smithi137.stdout:Ant: Apache Ant(TM) version 1.10.7 compiled on September 1 2019 2024-03-04T18:06:46.745 INFO:teuthology.orchestra.run.smithi137.stdout:JVM: 11.0.17 (Ubuntu 11.0.17+8-post-Ubuntu-1ubuntu222.04) 2024-03-04T18:06:46.745 INFO:teuthology.orchestra.run.smithi137.stdout:OS: Linux 5.15.0-97-generic amd64 2024-03-04T18:06:46.745 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:06:46.761 DEBUG:tasks.s3tests_java:S3 Tests Java: BEGIN 2024-03-04T18:06:46.761 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-03-04T18:06:46.761 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi137.front.sepia.ceph.com 2024-03-04T18:06:46.761 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-03-04T18:06:46.761 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi173.front.sepia.ceph.com 2024-03-04T18:06:46.761 INFO:tasks.s3tests_java:S3 Tests Java: Creating S3 users... 2024-03-04T18:06:46.955 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf is {'bucket_prefix': 'test-', 's3main': {'access_key': None, 'access_secret': None, 'region': 'mexico', 'endpoint': None, 'port': None, 'display_name': None, 'email': None, 'is_secure': None, 'SSE': 'AES256', 's3dir': 'folder', 'dir': '../data'}} 2024-03-04T18:06:46.955 DEBUG:tasks.s3tests_java:S3 Tests Java: Creating user tester.client.0 2024-03-04T18:06:46.955 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf[s3main] is {'access_key': 'PHFZVSZAUWZRHJCDCVFM', 'access_secret': 'J/Y36cTGPbsEKJbkPJ2Y1BMK0u26DyhPWg6UQGyRuOvY2nd96s0UAg==', 'region': 'mexico', 'endpoint': 'smithi137.front.sepia.ceph.com:443', 'port': 443, 'display_name': 'Ms. tester.client.0', 'email': 'tester.client.0_test@test.test', 'is_secure': True, 'SSE': 'AES256', 's3dir': 'folder', 'dir': '../data', 'user_id': 'tester.client.0', 'host': 'smithi137.front.sepia.ceph.com'} 2024-03-04T18:06:46.955 DEBUG:tasks.s3tests_java:S3 Tests Java: Setion, User = s3main, tester.client.0 2024-03-04T18:06:46.955 INFO:tasks.s3tests_java:['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '-n', 'client.0', 'user', 'create', '--uid', 'tester.client.0', '--display-name', 'Ms. tester.client.0', '--access-key', 'PHFZVSZAUWZRHJCDCVFM', '--secret', 'J/Y36cTGPbsEKJbkPJ2Y1BMK0u26DyhPWg6UQGyRuOvY2nd96s0UAg==', '--email', 'tester.client.0_test@test.test', '--cluster', 'ceph'] 2024-03-04T18:06:46.956 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid tester.client.0 --display-name 'Ms. tester.client.0' --access-key PHFZVSZAUWZRHJCDCVFM --secret J/Y36cTGPbsEKJbkPJ2Y1BMK0u26DyhPWg6UQGyRuOvY2nd96s0UAg== --email tester.client.0_test@test.test --cluster ceph 2024-03-04T18:06:46.996 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:46.996 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:47.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.034+0000 7fd5e7eec980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.038+0000 7fd5e7eec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.038+0000 7fd5e7eec980 20 realm 2024-03-04T18:06:47.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.038+0000 7fd5e7eec980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.038+0000 7fd5e7eec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.038+0000 7fd5e7eec980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:47.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.038+0000 7fd5e7eec980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.038+0000 7fd5e7eec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.038+0000 7fd5e7eec980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.038+0000 7fd5e7eec980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:47.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.038+0000 7fd5e7eec980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.042+0000 7fd5e7eec980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:47.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.042+0000 7fd5e7eec980 20 searching for the correct realm 2024-03-04T18:06:47.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.046+0000 7fd5e6205640 0 -- 172.21.15.137:0/774761986 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fd5e004d9e0 msgr2=0x7fd5e0058160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:47.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.262+0000 7fd5e6205640 0 -- 172.21.15.137:0/774761986 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x5606afb9bb70 msgr2=0x5606afb9e030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:47.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.270+0000 7fd5e7eec980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:47.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.270+0000 7fd5e7eec980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:47.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.270+0000 7fd5e7eec980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:47.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.270+0000 7fd5e7eec980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:47.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.270+0000 7fd5e7eec980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:47.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.270+0000 7fd5e7eec980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:47.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.270+0000 7fd5e7eec980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.270+0000 7fd5e7eec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.270+0000 7fd5e7eec980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.274+0000 7fd5e7eec980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:47.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.274+0000 7fd5e7eec980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.274+0000 7fd5e7eec980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.274+0000 7fd5e7eec980 20 zone default found 2024-03-04T18:06:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.274+0000 7fd5e7eec980 4 Realm: () 2024-03-04T18:06:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.274+0000 7fd5e7eec980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.274+0000 7fd5e7eec980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.274+0000 7fd5e7eec980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.274+0000 7fd5e7eec980 20 zonegroup default 2024-03-04T18:06:47.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.274+0000 7fd5e7eec980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.274+0000 7fd5e7eec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.274+0000 7fd5e7eec980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.274+0000 7fd5e7eec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.274+0000 7fd5e7eec980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.278+0000 7fd5e7eec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.278+0000 7fd5e7eec980 20 started sync module instance, tier type = 2024-03-04T18:06:47.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.278+0000 7fd5e7eec980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:47.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.302+0000 7fd5e7eec980 20 add_watcher() i=0 2024-03-04T18:06:47.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.302+0000 7fd5e7eec980 20 add_watcher() i=1 2024-03-04T18:06:47.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.302+0000 7fd5e7eec980 20 add_watcher() i=2 2024-03-04T18:06:47.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.302+0000 7fd5e7eec980 20 add_watcher() i=3 2024-03-04T18:06:47.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.302+0000 7fd5e7eec980 20 add_watcher() i=4 2024-03-04T18:06:47.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.302+0000 7fd5e7eec980 20 add_watcher() i=5 2024-03-04T18:06:47.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.302+0000 7fd5e7eec980 20 add_watcher() i=6 2024-03-04T18:06:47.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.302+0000 7fd5e7eec980 20 add_watcher() i=7 2024-03-04T18:06:47.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.302+0000 7fd5e7eec980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:47.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.306+0000 7fd4977fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:47.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.306+0000 7fd5e7eec980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:47.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.306+0000 7fd5e7eec980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:47.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.306+0000 7fd5e7eec980 5 note: GC not initialized 2024-03-04T18:06:47.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd4967fc640 20 reqs_thread_entry: start 2024-03-04T18:06:47.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:47.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 20 Filter name: none 2024-03-04T18:06:47.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd407fff640 20 reqs_thread_entry: start 2024-03-04T18:06:47.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2024-03-04T18:06:47.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x0 2024-03-04T18:06:47.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-03-04T18:06:47.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 10 cache get: name=default.rgw.meta+users.email+tester.client.0_test@test.test : miss 2024-03-04T18:06:47.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x0 2024-03-04T18:06:47.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 10 adding default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-03-04T18:06:47.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 10 cache get: name=default.rgw.meta+users.keys+PHFZVSZAUWZRHJCDCVFM : miss 2024-03-04T18:06:47.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 10 cache put: name=default.rgw.meta+users.keys+PHFZVSZAUWZRHJCDCVFM info.flags=0x0 2024-03-04T18:06:47.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 10 adding default.rgw.meta+users.keys+PHFZVSZAUWZRHJCDCVFM to cache LRU end 2024-03-04T18:06:47.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 10 cache get: name=default.rgw.meta+users.keys+PHFZVSZAUWZRHJCDCVFM : hit (negative entry) 2024-03-04T18:06:47.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.310+0000 7fd5e7eec980 10 cache get: name=default.rgw.meta+users.keys+PHFZVSZAUWZRHJCDCVFM : hit (negative entry) 2024-03-04T18:06:47.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.314+0000 7fd5e7eec980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2024-03-04T18:06:47.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.314+0000 7fd5e7eec980 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-03-04T18:06:47.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.314+0000 7fd5e7eec980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.uid:tester.client.0, ofs0, ns] 2024-03-04T18:06:47.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.314+0000 7fd4c97fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627832 cookie 94586718002240 notifier 6917 bl.length()=637 2024-03-04T18:06:47.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.314+0000 7fd4c97fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2024-03-04T18:06:47.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.314+0000 7fd4c97fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-03-04T18:06:47.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.322+0000 7fd5e7eec980 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2024-03-04T18:06:47.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.322+0000 7fd5e7eec980 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-03-04T18:06:47.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.322+0000 7fd5e7eec980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:tester.client.0_test@test.test, ofs0, ns] 2024-03-04T18:06:47.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.322+0000 7fd4c97fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627802 cookie 94586717992032 notifier 6917 bl.length()=200 2024-03-04T18:06:47.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.322+0000 7fd4c97fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2024-03-04T18:06:47.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.322+0000 7fd4c97fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-03-04T18:06:47.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.326+0000 7fd5e7eec980 10 cache put: name=default.rgw.meta+users.keys+PHFZVSZAUWZRHJCDCVFM info.flags=0x7 2024-03-04T18:06:47.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.326+0000 7fd5e7eec980 10 moving default.rgw.meta+users.keys+PHFZVSZAUWZRHJCDCVFM to cache LRU end 2024-03-04T18:06:47.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.326+0000 7fd5e7eec980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.keys:PHFZVSZAUWZRHJCDCVFM, ofs0, ns] 2024-03-04T18:06:47.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.326+0000 7fd4c97fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627885 cookie 94586717990816 notifier 6917 bl.length()=189 2024-03-04T18:06:47.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.326+0000 7fd4c97fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+PHFZVSZAUWZRHJCDCVFM info.flags=0x7 2024-03-04T18:06:47.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.326+0000 7fd4c97fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+PHFZVSZAUWZRHJCDCVFM to cache LRU end 2024-03-04T18:06:47.330 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-04T18:06:47.330 INFO:teuthology.orchestra.run.smithi137.stdout: "user_id": "tester.client.0", 2024-03-04T18:06:47.330 INFO:teuthology.orchestra.run.smithi137.stdout: "display_name": "Ms. tester.client.0", 2024-03-04T18:06:47.330 INFO:teuthology.orchestra.run.smithi137.stdout: "email": "tester.client.0_test@test.test", 2024-03-04T18:06:47.330 INFO:teuthology.orchestra.run.smithi137.stdout: "suspended": 0, 2024-03-04T18:06:47.330 INFO:teuthology.orchestra.run.smithi137.stdout: "max_buckets": 1000, 2024-03-04T18:06:47.330 INFO:teuthology.orchestra.run.smithi137.stdout: "subusers": [], 2024-03-04T18:06:47.330 INFO:teuthology.orchestra.run.smithi137.stdout: "keys": [ 2024-03-04T18:06:47.330 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:06:47.330 INFO:teuthology.orchestra.run.smithi137.stdout: "user": "tester.client.0", 2024-03-04T18:06:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "access_key": "PHFZVSZAUWZRHJCDCVFM", 2024-03-04T18:06:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "secret_key": "J/Y36cTGPbsEKJbkPJ2Y1BMK0u26DyhPWg6UQGyRuOvY2nd96s0UAg==", 2024-03-04T18:06:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "active": true 2024-03-04T18:06:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T18:06:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T18:06:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "swift_keys": [], 2024-03-04T18:06:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": [], 2024-03-04T18:06:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "op_mask": "read, write, delete", 2024-03-04T18:06:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "default_placement": "", 2024-03-04T18:06:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "default_storage_class": "", 2024-03-04T18:06:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "placement_tags": [], 2024-03-04T18:06:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bucket_quota": { 2024-03-04T18:06:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T18:06:47.331 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout: "user_quota": { 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout: "temp_url_keys": [], 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "rgw", 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout: "mfa_ids": [] 2024-03-04T18:06:47.332 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-04T18:06:47.333 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:06:47.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.330+0000 7fd4977fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:47.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.338+0000 7fd5e7eec980 20 remove_watcher() i=0 2024-03-04T18:06:47.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.338+0000 7fd5e7eec980 2 removed watcher, disabling cache 2024-03-04T18:06:47.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.338+0000 7fd5e7eec980 20 remove_watcher() i=1 2024-03-04T18:06:47.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.342+0000 7fd5e7eec980 20 remove_watcher() i=2 2024-03-04T18:06:47.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.342+0000 7fd5e7eec980 20 remove_watcher() i=3 2024-03-04T18:06:47.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.346+0000 7fd5e7eec980 20 remove_watcher() i=4 2024-03-04T18:06:47.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.350+0000 7fd5e7eec980 20 remove_watcher() i=5 2024-03-04T18:06:47.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.350+0000 7fd5e7eec980 20 remove_watcher() i=6 2024-03-04T18:06:47.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.354+0000 7fd5e7eec980 20 remove_watcher() i=7 2024-03-04T18:06:47.379 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T18:06:47.379 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf 2024-03-04T18:06:47.427 INFO:tasks.s3tests_java:S3 Tests Java: Running tests... 2024-03-04T18:06:47.427 DEBUG:teuthology.orchestra.run.smithi137:> cp /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf /home/ubuntu/cephtest/s3-tests-java/config.properties 2024-03-04T18:06:47.477 DEBUG:teuthology.orchestra.run.smithi137:> radosgw-admin gc process --include-all 2024-03-04T18:06:47.551 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:47.552 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:47.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.594+0000 7f4e100aa980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.598+0000 7f4e100aa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.598+0000 7f4e100aa980 20 realm 2024-03-04T18:06:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.598+0000 7f4e100aa980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.598+0000 7f4e100aa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.598+0000 7f4e100aa980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.598+0000 7f4e100aa980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.598+0000 7f4e100aa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.598+0000 7f4e100aa980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.602+0000 7f4e100aa980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:47.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.602+0000 7f4e100aa980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.602+0000 7f4e100aa980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:47.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.602+0000 7f4e100aa980 20 searching for the correct realm 2024-03-04T18:06:47.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.606+0000 7f4e0e3c3640 0 -- 172.21.15.137:0/1357078276 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x56541b1ee1f0 msgr2=0x56541b1f06b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:47.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.614+0000 7f4e0d3c1640 0 -- 172.21.15.137:0/1357078276 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x56541b1e9540 msgr2=0x56541b1eb950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:47.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:47.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:47.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:47.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:47.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:47.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:47.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:47.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:47.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 zone default found 2024-03-04T18:06:47.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 4 Realm: () 2024-03-04T18:06:47.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:47.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:47.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:47.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 zonegroup default 2024-03-04T18:06:47.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.622+0000 7f4e100aa980 20 rados->read ofs=0 len=0 2024-03-04T18:06:47.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.626+0000 7f4e100aa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:47.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.626+0000 7f4e100aa980 20 started sync module instance, tier type = 2024-03-04T18:06:47.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.626+0000 7f4e100aa980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:47.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.642+0000 7f4e100aa980 20 add_watcher() i=0 2024-03-04T18:06:47.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.642+0000 7f4e100aa980 20 add_watcher() i=1 2024-03-04T18:06:47.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.642+0000 7f4e100aa980 20 add_watcher() i=2 2024-03-04T18:06:47.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.642+0000 7f4e100aa980 20 add_watcher() i=3 2024-03-04T18:06:47.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.642+0000 7f4e100aa980 20 add_watcher() i=4 2024-03-04T18:06:47.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.642+0000 7f4e100aa980 20 add_watcher() i=5 2024-03-04T18:06:47.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.642+0000 7f4e100aa980 20 add_watcher() i=6 2024-03-04T18:06:47.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.646+0000 7f4e100aa980 20 add_watcher() i=7 2024-03-04T18:06:47.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.646+0000 7f4e100aa980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:47.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.650+0000 7f4cbf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:47.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.650+0000 7f4e100aa980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:47.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.650+0000 7f4e100aa980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.730+0000 7f4cbe7fc640 20 reqs_thread_entry: start 2024-03-04T18:06:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.730+0000 7f4e100aa980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.730+0000 7f4e100aa980 20 Filter name: none 2024-03-04T18:06:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.730+0000 7f4c2ffff640 20 reqs_thread_entry: start 2024-03-04T18:06:47.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.730+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:06:47.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.734+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.734+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.734+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:06:47.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.734+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_._zR58I_qyqqbRWVAUP-6Er01I-tUXyu_1_ff50590991ce275e96000db28804cbd9', time=2024-03-04T20:01:52.845671+0000, chain.objs.size()=1 2024-03-04T18:06:47.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.734+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_._zR58I_qyqqbRWVAUP-6Er01I-tUXyu_1 2024-03-04T18:06:47.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.738+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:47.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.742+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:06:47.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.746+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.746+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.746+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-03-04T18:06:47.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.746+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RTqqWOMCXYph639V0D8sAmr4hsu-qUk_1_a4e170753976efedd5e737f2855d154b', time=2024-03-04T20:01:52.790324+0000, chain.objs.size()=2 2024-03-04T18:06:47.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.746+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RTqqWOMCXYph639V0D8sAmr4hsu-qUk_0 2024-03-04T18:06:47.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.746+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RTqqWOMCXYph639V0D8sAmr4hsu-qUk_1 2024-03-04T18:06:47.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.746+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11571771833839545505', time=2024-03-04T20:01:52.964899+0000, chain.objs.size()=3 2024-03-04T18:06:47.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.746+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.1 2024-03-04T18:06:47.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.746+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.2 2024-03-04T18:06:47.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.746+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.3 2024-03-04T18:06:47.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.746+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='_LJf1bjafIKSjGAwnWqz-qah9JjzNfwm', time=2024-03-04T20:01:52.983182+0000, chain.objs.size()=1 2024-03-04T18:06:47.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.746+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.-8XtQ647UsHdVRWlV1knGZWxVbQkDhg_0 2024-03-04T18:06:47.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.750+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:47.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.754+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:06:47.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.758+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.758+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.758+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.762+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:06:47.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.762+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.766+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.766+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-03-04T18:06:47.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.766+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10989446842724053574', time=2024-03-04T20:01:52.766911+0000, chain.objs.size()=2 2024-03-04T18:06:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.766+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RTqqWOMCXYph639V0D8sAmr4hsu-qUk_0 2024-03-04T18:06:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.766+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RTqqWOMCXYph639V0D8sAmr4hsu-qUk_1 2024-03-04T18:06:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.766+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='_TxG7S7fyR9w-UlJvu2pQH_p6HWflGks', time=2024-03-04T20:01:52.833701+0000, chain.objs.size()=2 2024-03-04T18:06:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.766+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.josUa4jc9O-fQ447L6UEb10MtPW9smw_0 2024-03-04T18:06:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.766+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.josUa4jc9O-fQ447L6UEb10MtPW9smw_1 2024-03-04T18:06:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.766+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.1_1_4867b1abbb328ab64766375e7f6d0f42', time=2024-03-04T20:01:52.895296+0000, chain.objs.size()=5 2024-03-04T18:06:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.766+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.1 2024-03-04T18:06:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.766+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.1_1 2024-03-04T18:06:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.766+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.2 2024-03-04T18:06:47.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.766+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.2_1 2024-03-04T18:06:47.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.766+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.3 2024-03-04T18:06:47.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.774+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:47.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.778+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:06:47.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.782+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.782+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.782+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:06:47.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.782+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4585518418313471972', time=2024-03-04T20:01:53.033708+0000, chain.objs.size()=5 2024-03-04T18:06:47.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.782+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-FROZEN.2~-8noU2j1nj9CEob3DnwSrTLOYWTYQGl.1 2024-03-04T18:06:47.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.782+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__shadow_foo-FROZEN.2~-8noU2j1nj9CEob3DnwSrTLOYWTYQGl.1_1 2024-03-04T18:06:47.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.782+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-FROZEN.2~-8noU2j1nj9CEob3DnwSrTLOYWTYQGl.2 2024-03-04T18:06:47.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.782+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__shadow_foo-FROZEN.2~-8noU2j1nj9CEob3DnwSrTLOYWTYQGl.2_1 2024-03-04T18:06:47.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.782+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-FROZEN.2~-8noU2j1nj9CEob3DnwSrTLOYWTYQGl.3 2024-03-04T18:06:47.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.790+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:47.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.794+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:06:47.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.798+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.798+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.802+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:06:47.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.802+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='_rwJSNkoRMMIK1Lo9Vz5Bw20kCoZYvKE', time=2024-03-04T20:01:52.779032+0000, chain.objs.size()=2 2024-03-04T18:06:47.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.802+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.caDq23bEIJ1nynWJcaF8C308__7ZFv2_0 2024-03-04T18:06:47.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.802+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.caDq23bEIJ1nynWJcaF8C308__7ZFv2_1 2024-03-04T18:06:47.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.802+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.1042772702181890393', time=2024-03-04T20:01:52.924763+0000, chain.objs.size()=5 2024-03-04T18:06:47.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.802+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.1 2024-03-04T18:06:47.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.802+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.1_1 2024-03-04T18:06:47.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.802+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.2 2024-03-04T18:06:47.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.802+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.2_1 2024-03-04T18:06:47.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.802+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.3 2024-03-04T18:06:47.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.802+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:47.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.810+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:06:47.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.814+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.814+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.814+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:06:47.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.814+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15912843451726240125', time=2024-03-04T20:01:52.886060+0000, chain.objs.size()=5 2024-03-04T18:06:47.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.814+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.1 2024-03-04T18:06:47.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.814+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.1_1 2024-03-04T18:06:47.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.814+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.2 2024-03-04T18:06:47.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.814+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.2_1 2024-03-04T18:06:47.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.814+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-FROZEN.2~RcnfevP7HFdDMuIeZIAlPDR5oxKcNVA.3 2024-03-04T18:06:47.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.818+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:47.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.818+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:06:47.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.822+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.826+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.826+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:06:47.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.826+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.4jckuAtqyME-A0HXCRRmp5A1eKNUgIp_1_7fdc11563e2571435be2c59c8c380d8b', time=2024-03-04T20:01:52.733859+0000, chain.objs.size()=2 2024-03-04T18:06:47.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.826+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.4jckuAtqyME-A0HXCRRmp5A1eKNUgIp_0 2024-03-04T18:06:47.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.826+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.4jckuAtqyME-A0HXCRRmp5A1eKNUgIp_1 2024-03-04T18:06:47.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.826+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:47.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.834+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:06:47.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.838+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.838+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.838+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.842+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:06:47.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.846+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.846+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.846+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.850+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:06:47.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.850+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.854+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.854+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:06:47.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.854+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.3199931641681391603', time=2024-03-04T20:01:52.557505+0000, chain.objs.size()=4 2024-03-04T18:06:47.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.854+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12__multipart_foo.2~R7XuWjpGEIqxR6TM1E-GwEOzOKD1pdy.1 2024-03-04T18:06:47.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.854+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12__multipart_foo.2~R7XuWjpGEIqxR6TM1E-GwEOzOKD1pdy.2 2024-03-04T18:06:47.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.854+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12__multipart_foo.2~R7XuWjpGEIqxR6TM1E-GwEOzOKD1pdy.3 2024-03-04T18:06:47.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.854+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12__multipart_foo.2~R7XuWjpGEIqxR6TM1E-GwEOzOKD1pdy.4 2024-03-04T18:06:47.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.858+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:47.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.862+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:06:47.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.866+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.866+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.870+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-03-04T18:06:47.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.870+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13719652918541394405', time=2024-03-04T20:01:52.640259+0000, chain.objs.size()=4 2024-03-04T18:06:47.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.870+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11__multipart_foo.2~MPH5BNCPADH4YVw1XTdHNJTjFnERwPX.1 2024-03-04T18:06:47.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.870+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11__multipart_foo.2~MPH5BNCPADH4YVw1XTdHNJTjFnERwPX.2 2024-03-04T18:06:47.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.870+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11__multipart_foo.2~MPH5BNCPADH4YVw1XTdHNJTjFnERwPX.3 2024-03-04T18:06:47.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.870+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11__multipart_foo.2~MPH5BNCPADH4YVw1XTdHNJTjFnERwPX.4 2024-03-04T18:06:47.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.870+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='_NlKJBoXIrQHizf0-j5ZnMyRCi36PbB5', time=2024-03-04T20:01:52.905839+0000, chain.objs.size()=3 2024-03-04T18:06:47.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.870+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.TSy8ST9GOrEzFO--nQ3CMdbrMD__UlR_0 2024-03-04T18:06:47.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.870+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.TSy8ST9GOrEzFO--nQ3CMdbrMD__UlR_1 2024-03-04T18:06:47.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.870+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.TSy8ST9GOrEzFO--nQ3CMdbrMD__UlR_2 2024-03-04T18:06:47.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.870+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17231879429835817372', time=2024-03-04T20:01:53.056880+0000, chain.objs.size()=3 2024-03-04T18:06:47.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.870+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-STANDARD.2~J4gplH8e0-_OUb8Iirm60u9-a4HqabO.1 2024-03-04T18:06:47.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.870+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-STANDARD.2~J4gplH8e0-_OUb8Iirm60u9-a4HqabO.2 2024-03-04T18:06:47.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.870+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-STANDARD.2~J4gplH8e0-_OUb8Iirm60u9-a4HqabO.3 2024-03-04T18:06:47.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.878+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:47.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.882+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:06:47.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.886+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.886+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.890+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:06:47.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.890+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='_90hHByb_ggy_cVbPRpMGjeMwyLzzKLs', time=2024-03-04T20:01:52.756286+0000, chain.objs.size()=1 2024-03-04T18:06:47.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.890+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.NIkysXa0IUNScOZUuV-utKeAhdzDjij_1 2024-03-04T18:06:47.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.890+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:47.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.898+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:06:47.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.902+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.902+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.902+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:06:47.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.902+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10390100625577012679', time=2024-03-04T20:01:53.101114+0000, chain.objs.size()=3 2024-03-04T18:06:47.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.902+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14__shadow_.4YDwkBT-ECBPSaihDhWwP99qJf6xg1o_0 2024-03-04T18:06:47.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.902+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14__shadow_.4YDwkBT-ECBPSaihDhWwP99qJf6xg1o_1 2024-03-04T18:06:47.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.902+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14__shadow_.4YDwkBT-ECBPSaihDhWwP99qJf6xg1o_2 2024-03-04T18:06:47.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.910+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:47.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.914+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:06:47.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.918+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.918+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.922+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:06:47.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.922+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='_Z-8Jc8rUmqw3-MYkgcKZPVy3x6dz268', time=2024-03-04T20:01:52.932486+0000, chain.objs.size()=3 2024-03-04T18:06:47.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.922+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.xOSVFtoJu6VjTAIM3u8UDX-vIOFPOPb_0 2024-03-04T18:06:47.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.922+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.xOSVFtoJu6VjTAIM3u8UDX-vIOFPOPb_1 2024-03-04T18:06:47.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.922+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.xOSVFtoJu6VjTAIM3u8UDX-vIOFPOPb_2 2024-03-04T18:06:47.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.926+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:47.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.930+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:06:47.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.934+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.934+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.934+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.938+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:06:47.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.938+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.942+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.942+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.946+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:06:47.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.946+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.946+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.950+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.950+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:06:47.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.954+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.954+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.958+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.958+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:06:47.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.962+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.962+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.966+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.966+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:06:47.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.970+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.970+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.974+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:06:47.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.974+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2697193294386511849', time=2024-03-04T20:01:52.724140+0000, chain.objs.size()=2 2024-03-04T18:06:47.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.974+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.4jckuAtqyME-A0HXCRRmp5A1eKNUgIp_0 2024-03-04T18:06:47.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.974+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.4jckuAtqyME-A0HXCRRmp5A1eKNUgIp_1 2024-03-04T18:06:47.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.974+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10976351442943365117', time=2024-03-04T20:01:52.816183+0000, chain.objs.size()=1 2024-03-04T18:06:47.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.974+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_._zR58I_qyqqbRWVAUP-6Er01I-tUXyu_1 2024-03-04T18:06:47.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.974+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:47.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.982+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:06:47.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.986+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:47.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.986+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:47.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.986+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:06:47.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.986+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='_p1NtDaqrf3bTUBT6k1CC3gsPW1FuokH', time=2024-03-04T20:01:52.914452+0000, chain.objs.size()=2 2024-03-04T18:06:47.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.986+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.SROXVv7gX2el9hNB4DNUpkm2BDEkY8Y_1 2024-03-04T18:06:47.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.986+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.SROXVv7gX2el9hNB4DNUpkm2BDEkY8Y_2 2024-03-04T18:06:47.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.994+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:47.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.998+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:06:47.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:47.998+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-03-04T18:06:48.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16904344668637114563', time=2024-03-04T20:01:52.683351+0000, chain.objs.size()=7 2024-03-04T18:06:48.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10__multipart_foo.2~RA4PcczPDlMtdmBtXJzZm2-Cv1sKurx.1 2024-03-04T18:06:48.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10__shadow_foo.2~RA4PcczPDlMtdmBtXJzZm2-Cv1sKurx.1_1 2024-03-04T18:06:48.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10__multipart_foo.2~RA4PcczPDlMtdmBtXJzZm2-Cv1sKurx.2 2024-03-04T18:06:48.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10__shadow_foo.2~RA4PcczPDlMtdmBtXJzZm2-Cv1sKurx.2_1 2024-03-04T18:06:48.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10__multipart_foo.2~RA4PcczPDlMtdmBtXJzZm2-Cv1sKurx.3 2024-03-04T18:06:48.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10__shadow_foo.2~RA4PcczPDlMtdmBtXJzZm2-Cv1sKurx.3_1 2024-03-04T18:06:48.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.10__multipart_foo.2~RA4PcczPDlMtdmBtXJzZm2-Cv1sKurx.4 2024-03-04T18:06:48.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='_qegkQBlWqlwZgv-gauKLFcpFmReMnCL', time=2024-03-04T20:01:52.744249+0000, chain.objs.size()=2 2024-03-04T18:06:48.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RVCQ5x81jkkIXzZWFqabLMUVlfFF7s1_0 2024-03-04T18:06:48.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.RVCQ5x81jkkIXzZWFqabLMUVlfFF7s1_1 2024-03-04T18:06:48.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='_J-K_7yuwUD2N9ieyruKnmaglaFbqRPH', time=2024-03-04T20:01:52.803999+0000, chain.objs.size()=1 2024-03-04T18:06:48.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.002+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.K_N1fudpCWjWZAYttGsRM9SjO805mRE_1 2024-03-04T18:06:48.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.014+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:48.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.018+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:06:48.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.022+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.022+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.022+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:06:48.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.022+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.874685326992247852', time=2024-03-04T20:01:52.592935+0000, chain.objs.size()=4 2024-03-04T18:06:48.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.022+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13__multipart_foo.2~r9w9SisoYP9sAzMu8TPK7loFOqHcPLA.1 2024-03-04T18:06:48.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.022+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13__multipart_foo.2~r9w9SisoYP9sAzMu8TPK7loFOqHcPLA.2 2024-03-04T18:06:48.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.022+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13__multipart_foo.2~r9w9SisoYP9sAzMu8TPK7loFOqHcPLA.3 2024-03-04T18:06:48.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.022+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13__multipart_foo.2~r9w9SisoYP9sAzMu8TPK7loFOqHcPLA.4 2024-03-04T18:06:48.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.026+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:48.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.030+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:06:48.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.034+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.038+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.038+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:06:48.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.038+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.1_1_b4a185200b990cf923cf0a61d75c7a91', time=2024-03-04T20:01:52.944705+0000, chain.objs.size()=5 2024-03-04T18:06:48.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.038+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.1 2024-03-04T18:06:48.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.038+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.1_1 2024-03-04T18:06:48.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.038+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.2 2024-03-04T18:06:48.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.038+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.2_1 2024-03-04T18:06:48.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.038+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-LUKEWARM.2~DkEZXvhbsnUQaQ1l3uqTP8TfMloDFlC.3 2024-03-04T18:06:48.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.042+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:48.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.046+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:06:48.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.050+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.054+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.054+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.058+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:06:48.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.058+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.062+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.062+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.066+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:06:48.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.066+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.070+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.070+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.074+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:06:48.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.078+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.078+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.078+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:06:48.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.078+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='_w-EYfF1oQ5unXdigbGQeEIZTTcTPrk0', time=2024-03-04T20:01:52.974042+0000, chain.objs.size()=1 2024-03-04T18:06:48.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.078+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.m-iJ5fVdcIxnFHP5ZgRESWTgprLMnIL_0 2024-03-04T18:06:48.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.082+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:48.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.090+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:06:48.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.094+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.094+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.094+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:06:48.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.094+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='_Q3ZWNU-Pw_prsppMiTcDLtG-KotW95S', time=2024-03-04T20:01:52.955886+0000, chain.objs.size()=2 2024-03-04T18:06:48.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.094+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.E9aNPC3qlrcDbdCt2UPifaaDXgBq15u_1 2024-03-04T18:06:48.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.094+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__shadow_.E9aNPC3qlrcDbdCt2UPifaaDXgBq15u_2 2024-03-04T18:06:48.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.102+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:48.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.106+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:06:48.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.114+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.114+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.114+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:06:48.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.114+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='_QwCP8FLLMzZYweg_DuaRZO7VcLpDbdu', time=2024-03-04T20:01:52.823920+0000, chain.objs.size()=2 2024-03-04T18:06:48.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.114+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.loKTFsbdHc5Wwog7N5txaJ8ezlzUf3Z_0 2024-03-04T18:06:48.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.114+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16__shadow_.loKTFsbdHc5Wwog7N5txaJ8ezlzUf3Z_1 2024-03-04T18:06:48.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.114+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.2_e455ae1c2e5a219f513d7aaddc21adc8', time=2024-03-04T20:01:52.990802+0000, chain.objs.size()=3 2024-03-04T18:06:48.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.114+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.1 2024-03-04T18:06:48.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.114+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.2 2024-03-04T18:06:48.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.114+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17__multipart_foo-STANDARD.2~x0ASpHa8jOEaUS5lM8iHaviMNiF2fux.3 2024-03-04T18:06:48.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.122+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:48.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.126+0000 7f4e100aa980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:06:48.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.130+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.134+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.134+0000 7f4e100aa980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:06:48.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.134+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.2499779498388887256', time=2024-03-04T20:01:53.046270+0000, chain.objs.size()=5 2024-03-04T18:06:48.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.134+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-LUKEWARM.2~OYMnPgqQAanTFlMmjgFQjXzvKfR2zVn.1 2024-03-04T18:06:48.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.134+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__shadow_foo-LUKEWARM.2~OYMnPgqQAanTFlMmjgFQjXzvKfR2zVn.1_1 2024-03-04T18:06:48.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.134+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-LUKEWARM.2~OYMnPgqQAanTFlMmjgFQjXzvKfR2zVn.2 2024-03-04T18:06:48.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.134+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__shadow_foo-LUKEWARM.2~OYMnPgqQAanTFlMmjgFQjXzvKfR2zVn.2_1 2024-03-04T18:06:48.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.134+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15__multipart_foo-LUKEWARM.2~OYMnPgqQAanTFlMmjgFQjXzvKfR2zVn.3 2024-03-04T18:06:48.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.134+0000 7f4e100aa980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16122458151288315120', time=2024-03-04T20:01:53.111619+0000, chain.objs.size()=3 2024-03-04T18:06:48.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.134+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14__shadow_.xRJ6_oVzZJ4Pc0ktCPGTWZFl7mDxVSU_0 2024-03-04T18:06:48.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.134+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14__shadow_.xRJ6_oVzZJ4Pc0ktCPGTWZFl7mDxVSU_1 2024-03-04T18:06:48.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.134+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14__shadow_.xRJ6_oVzZJ4Pc0ktCPGTWZFl7mDxVSU_2 2024-03-04T18:06:48.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.146+0000 7f4e100aa980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:06:48.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.154+0000 7f4cbf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:48.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.158+0000 7f4e100aa980 20 remove_watcher() i=0 2024-03-04T18:06:48.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.158+0000 7f4e100aa980 2 removed watcher, disabling cache 2024-03-04T18:06:48.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.162+0000 7f4e100aa980 20 remove_watcher() i=1 2024-03-04T18:06:48.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.166+0000 7f4e100aa980 20 remove_watcher() i=2 2024-03-04T18:06:48.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.166+0000 7f4e100aa980 20 remove_watcher() i=3 2024-03-04T18:06:48.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.170+0000 7f4e100aa980 20 remove_watcher() i=4 2024-03-04T18:06:48.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.170+0000 7f4e100aa980 20 remove_watcher() i=5 2024-03-04T18:06:48.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.174+0000 7f4e100aa980 20 remove_watcher() i=6 2024-03-04T18:06:48.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.178+0000 7f4e100aa980 20 remove_watcher() i=7 2024-03-04T18:06:48.198 DEBUG:teuthology.orchestra.run.smithi137:> radosgw-admin gc process --include-all 2024-03-04T18:06:48.272 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:06:48.272 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:06:48.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.314+0000 7f2367512980 20 rados->read ofs=0 len=0 2024-03-04T18:06:48.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.318+0000 7f2367512980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:48.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.318+0000 7f2367512980 20 realm 2024-03-04T18:06:48.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.318+0000 7f2367512980 20 rados->read ofs=0 len=0 2024-03-04T18:06:48.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.318+0000 7f2367512980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:48.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.318+0000 7f2367512980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:06:48.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.318+0000 7f2367512980 20 rados->read ofs=0 len=0 2024-03-04T18:06:48.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.318+0000 7f2367512980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:48.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.318+0000 7f2367512980 20 rados->read ofs=0 len=0 2024-03-04T18:06:48.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.318+0000 7f2367512980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:48.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.318+0000 7f2367512980 20 rados->read ofs=0 len=0 2024-03-04T18:06:48.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.322+0000 7f2367512980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:06:48.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.322+0000 7f2367512980 20 searching for the correct realm 2024-03-04T18:06:48.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.326+0000 7f236582b640 0 -- 172.21.15.137:0/2780219941 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f23600fde30 msgr2=0x7f2360122120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:06:48.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.542+0000 7f236582b640 0 -- 172.21.15.137:0/2780219941 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x556e31c27b30 msgr2=0x556e31c29ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:06:48.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.550+0000 7f2367512980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:06:48.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.550+0000 7f2367512980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:06:48.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.550+0000 7f2367512980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:06:48.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.550+0000 7f2367512980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:06:48.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.550+0000 7f2367512980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:06:48.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.550+0000 7f2367512980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:06:48.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.550+0000 7f2367512980 20 rados->read ofs=0 len=0 2024-03-04T18:06:48.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.550+0000 7f2367512980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:48.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.550+0000 7f2367512980 20 rados->read ofs=0 len=0 2024-03-04T18:06:48.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.550+0000 7f2367512980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:06:48.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.550+0000 7f2367512980 20 rados->read ofs=0 len=0 2024-03-04T18:06:48.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:06:48.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 20 zone default found 2024-03-04T18:06:48.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 4 Realm: () 2024-03-04T18:06:48.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:06:48.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:06:48.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:06:48.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 20 zonegroup default 2024-03-04T18:06:48.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 20 rados->read ofs=0 len=0 2024-03-04T18:06:48.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:48.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 20 rados->read ofs=0 len=0 2024-03-04T18:06:48.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:48.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 20 rados->read ofs=0 len=0 2024-03-04T18:06:48.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:06:48.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 20 started sync module instance, tier type = 2024-03-04T18:06:48.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.554+0000 7f2367512980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:06:48.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.578+0000 7f2367512980 20 add_watcher() i=0 2024-03-04T18:06:48.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.578+0000 7f2367512980 20 add_watcher() i=1 2024-03-04T18:06:48.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.578+0000 7f2367512980 20 add_watcher() i=2 2024-03-04T18:06:48.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.578+0000 7f2367512980 20 add_watcher() i=3 2024-03-04T18:06:48.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.578+0000 7f2367512980 20 add_watcher() i=4 2024-03-04T18:06:48.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.578+0000 7f2367512980 20 add_watcher() i=5 2024-03-04T18:06:48.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.578+0000 7f2367512980 20 add_watcher() i=6 2024-03-04T18:06:48.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.582+0000 7f2367512980 20 add_watcher() i=7 2024-03-04T18:06:48.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.582+0000 7f2367512980 2 all 8 watchers are set, enabling cache 2024-03-04T18:06:48.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.586+0000 7f221b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:48.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.586+0000 7f2367512980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:06:48.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.586+0000 7f2367512980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:06:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.682+0000 7f221a7e4640 20 reqs_thread_entry: start 2024-03-04T18:06:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.682+0000 7f2367512980 20 init_complete bucket index max shards: 1 2024-03-04T18:06:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.682+0000 7f2367512980 20 Filter name: none 2024-03-04T18:06:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.682+0000 7f218bfff640 20 reqs_thread_entry: start 2024-03-04T18:06:48.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.682+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:06:48.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.686+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.686+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.686+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.690+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:06:48.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.690+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.690+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.694+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.694+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:06:48.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.698+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.698+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.698+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.702+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:06:48.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.702+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.706+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.706+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.706+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:06:48.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.710+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.710+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.714+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.714+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:06:48.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.718+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.718+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.722+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.722+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:06:48.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.726+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.726+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.726+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.730+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:06:48.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.730+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.734+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.734+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.734+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:06:48.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.738+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.738+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.742+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.742+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:06:48.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.746+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.746+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.746+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.750+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:06:48.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.754+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.754+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.754+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.758+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:06:48.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.762+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.766+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.766+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.766+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:06:48.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.770+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.770+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.770+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.774+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:06:48.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.778+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.778+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.782+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.782+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:06:48.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.786+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.786+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.790+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.790+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:06:48.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.794+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.798+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.798+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.798+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:06:48.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.802+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.802+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.802+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.806+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:06:48.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.810+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.810+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.810+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.814+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:06:48.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.818+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.818+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.822+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.822+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:06:48.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.826+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.830+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.830+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.830+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:06:48.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.834+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.834+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.834+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.838+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:06:48.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.838+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.842+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.842+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.842+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:06:48.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.846+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.850+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.850+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.850+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:06:48.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.858+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.858+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.858+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.862+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:06:48.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.866+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.866+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.866+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.870+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:06:48.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.874+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.874+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.874+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.878+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:06:48.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.882+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.882+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.882+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.886+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:06:48.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.890+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.890+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.894+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.894+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:06:48.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.898+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.898+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.902+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.902+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:06:48.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.906+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.910+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.910+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.914+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:06:48.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.918+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.918+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.918+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.922+0000 7f2367512980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.926+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.926+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.926+0000 7f2367512980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.934+0000 7f221b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.938+0000 7f2367512980 20 remove_watcher() i=0 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.938+0000 7f2367512980 2 removed watcher, disabling cache 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.938+0000 7f2367512980 20 remove_watcher() i=1 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.942+0000 7f2367512980 20 remove_watcher() i=2 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.946+0000 7f2367512980 20 remove_watcher() i=3 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.946+0000 7f2367512980 20 remove_watcher() i=4 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.950+0000 7f2367512980 20 remove_watcher() i=5 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.954+0000 7f2367512980 20 remove_watcher() i=6 2024-03-04T18:06:48.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:06:48.954+0000 7f2367512980 20 remove_watcher() i=7 2024-03-04T18:06:48.974 DEBUG:teuthology.orchestra.run.smithi137:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test 2024-03-04T18:06:49.567 INFO:teuthology.orchestra.run.smithi137.stdout:Starting a Gradle Daemon (subsequent builds will be faster) 2024-03-04T18:06:54.368 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :clean UP-TO-DATE 2024-03-04T18:07:14.267 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :compileJava 2024-03-04T18:07:14.267 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :processResources 2024-03-04T18:07:14.267 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :classes 2024-03-04T18:07:15.166 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:07:15.167 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :compileTestJava 2024-03-04T18:07:15.167 INFO:teuthology.orchestra.run.smithi137.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-03-04T18:07:15.167 INFO:teuthology.orchestra.run.smithi137.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-03-04T18:07:15.168 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:07:15.168 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :processTestResources NO-SOURCE 2024-03-04T18:07:15.168 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :testClasses 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :test 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:07:21.767 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 STARTED 2024-03-04T18:07:34.167 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:07:34.167 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 PASSED 2024-03-04T18:07:34.666 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:07:34.666 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 STARTED 2024-03-04T18:07:34.866 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:07:34.866 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 PASSED 2024-03-04T18:07:35.466 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:07:35.466 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 STARTED 2024-03-04T18:07:56.967 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:07:56.967 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 PASSED 2024-03-04T18:07:57.466 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:07:57.466 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 STARTED 2024-03-04T18:07:57.566 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:07:57.566 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 PASSED 2024-03-04T18:07:57.666 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:07:57.666 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 STARTED 2024-03-04T18:07:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:07:57.866 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 PASSED 2024-03-04T18:07:58.366 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:07:58.366 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 STARTED 2024-03-04T18:08:12.266 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:08:12.267 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 PASSED 2024-03-04T18:08:13.266 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:08:13.266 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STARTED 2024-03-04T18:08:13.666 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:08:13.666 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STANDARD_OUT 2024-03-04T18:08:13.667 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04 18:08:13,603 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-03-04T18:09:09.167 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:09:09.168 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 PASSED 2024-03-04T18:09:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:09:10.167 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 STARTED 2024-03-04T18:09:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:09:10.267 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 PASSED 2024-03-04T18:09:10.767 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:09:10.767 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 STARTED 2024-03-04T18:09:10.967 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:09:10.967 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 PASSED 2024-03-04T18:09:11.867 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:09:11.867 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 STARTED 2024-03-04T18:09:24.667 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:09:24.667 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 PASSED 2024-03-04T18:09:25.267 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:09:25.267 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 STARTED 2024-03-04T18:09:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:09:25.367 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 PASSED 2024-03-04T18:09:25.567 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:09:25.567 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STARTED 2024-03-04T18:09:40.769 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:09:40.769 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STANDARD_OUT 2024-03-04T18:09:40.769 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04 18:09:40,706 [s3-transfer-manager-worker-3] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-03-04T18:09:40.769 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04 18:09:40,736 [s3-transfer-manager-worker-5] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-03-04T18:09:40.769 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04 18:09:40,747 [s3-transfer-manager-worker-2] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-03-04T18:09:40.769 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04 18:09:40,752 [s3-transfer-manager-worker-4] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-03-04T18:09:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:09:41.067 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 PASSED 2024-03-04T18:09:42.568 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:09:42.568 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests STANDARD_OUT 2024-03-04T18:09:42.568 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04 18:09:42,532 [s3-transfer-manager-worker-9] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-03-04T18:09:42.568 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04 18:09:42,545 [s3-transfer-manager-worker-8] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-03-04T18:09:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:09:42.867 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 STARTED 2024-03-04T18:10:08.368 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:10:08.368 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 PASSED 2024-03-04T18:10:09.267 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:10:09.267 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 STARTED 2024-03-04T18:10:15.467 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:10:15.467 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 PASSED 2024-03-04T18:10:16.067 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:10:16.067 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 STARTED 2024-03-04T18:13:24.168 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:13:24.169 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 PASSED 2024-03-04T18:13:24.767 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:13:24.767 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 STARTED 2024-03-04T18:13:41.267 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:13:41.268 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 PASSED 2024-03-04T18:13:41.767 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:13:41.767 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 STARTED 2024-03-04T18:14:03.568 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:03.568 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 PASSED 2024-03-04T18:14:04.167 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:04.167 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 STARTED 2024-03-04T18:14:08.568 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:08.568 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 PASSED 2024-03-04T18:14:17.468 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:17.468 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 STARTED 2024-03-04T18:14:18.268 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:18.268 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 PASSED 2024-03-04T18:14:22.168 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:22.168 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 STARTED 2024-03-04T18:14:22.268 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:22.268 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 PASSED 2024-03-04T18:14:22.868 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:22.868 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 STARTED 2024-03-04T18:14:23.067 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:23.067 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 PASSED 2024-03-04T18:14:23.668 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:23.668 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 STARTED 2024-03-04T18:14:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:23.868 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 PASSED 2024-03-04T18:14:24.468 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:24.468 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 STARTED 2024-03-04T18:14:24.668 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:24.668 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 PASSED 2024-03-04T18:14:25.267 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:25.268 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 STARTED 2024-03-04T18:14:25.467 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:25.468 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 PASSED 2024-03-04T18:14:26.067 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:26.068 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 STARTED 2024-03-04T18:14:26.268 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:26.268 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 PASSED 2024-03-04T18:14:26.667 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:26.668 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 STARTED 2024-03-04T18:14:55.768 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:55.768 INFO:teuthology.orchestra.run.smithi137.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 PASSED 2024-03-04T18:14:56.468 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:56.468 INFO:teuthology.orchestra.run.smithi137.stdout:---------------------------------------------------------------------- 2024-03-04T18:14:56.468 INFO:teuthology.orchestra.run.smithi137.stdout:| Results: SUCCESS (26 tests, 26 successes, 0 failures, 0 skipped) | 2024-03-04T18:14:56.468 INFO:teuthology.orchestra.run.smithi137.stdout:---------------------------------------------------------------------- 2024-03-04T18:14:56.497 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:56.497 INFO:teuthology.orchestra.run.smithi137.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-03-04T18:14:56.497 INFO:teuthology.orchestra.run.smithi137.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-03-04T18:14:56.497 INFO:teuthology.orchestra.run.smithi137.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-03-04T18:14:56.497 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:14:56.498 INFO:teuthology.orchestra.run.smithi137.stdout:BUILD SUCCESSFUL in 8m 7s 2024-03-04T18:14:56.498 INFO:teuthology.orchestra.run.smithi137.stdout:5 actionable tasks: 4 executed, 1 up-to-date 2024-03-04T18:14:56.826 DEBUG:teuthology.orchestra.run.smithi137:> radosgw-admin gc process --include-all 2024-03-04T18:14:56.857 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:14:56.857 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:14:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:56.900+0000 7f45a8c48980 20 rados->read ofs=0 len=0 2024-03-04T18:14:56.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:56.900+0000 7f45a8c48980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:56.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:56.900+0000 7f45a8c48980 20 realm 2024-03-04T18:14:56.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:56.900+0000 7f45a8c48980 20 rados->read ofs=0 len=0 2024-03-04T18:14:56.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:56.900+0000 7f45a8c48980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:56.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:56.900+0000 7f45a8c48980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:14:56.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:56.900+0000 7f45a8c48980 20 rados->read ofs=0 len=0 2024-03-04T18:14:56.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:56.900+0000 7f45a8c48980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:56.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:56.900+0000 7f45a8c48980 20 rados->read ofs=0 len=0 2024-03-04T18:14:56.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:56.904+0000 7f45a8c48980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:14:56.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:56.904+0000 7f45a8c48980 20 rados->read ofs=0 len=0 2024-03-04T18:14:56.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:56.904+0000 7f45a8c48980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:14:56.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:56.904+0000 7f45a8c48980 20 searching for the correct realm 2024-03-04T18:14:56.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:56.908+0000 7f45a6f61640 0 -- 172.21.15.137:0/4150680133 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f45a002d910 msgr2=0x7f45a000f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:14:57.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.124+0000 7f45a6f61640 0 -- 172.21.15.137:0/4150680133 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x563ccb987660 msgr2=0x563ccb989b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:14:57.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.132+0000 7f45a8c48980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:14:57.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.132+0000 7f45a8c48980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:14:57.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.132+0000 7f45a8c48980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:14:57.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.132+0000 7f45a8c48980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:14:57.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.132+0000 7f45a8c48980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:14:57.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.132+0000 7f45a8c48980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:14:57.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.132+0000 7f45a8c48980 20 rados->read ofs=0 len=0 2024-03-04T18:14:57.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.132+0000 7f45a8c48980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:57.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 20 rados->read ofs=0 len=0 2024-03-04T18:14:57.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:14:57.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 20 rados->read ofs=0 len=0 2024-03-04T18:14:57.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:14:57.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 20 zone default found 2024-03-04T18:14:57.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 4 Realm: () 2024-03-04T18:14:57.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:14:57.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:14:57.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:14:57.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 20 zonegroup default 2024-03-04T18:14:57.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 20 rados->read ofs=0 len=0 2024-03-04T18:14:57.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:57.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 20 rados->read ofs=0 len=0 2024-03-04T18:14:57.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:57.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 20 rados->read ofs=0 len=0 2024-03-04T18:14:57.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:57.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 20 started sync module instance, tier type = 2024-03-04T18:14:57.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.136+0000 7f45a8c48980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:14:57.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.160+0000 7f45a8c48980 20 add_watcher() i=0 2024-03-04T18:14:57.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.160+0000 7f45a8c48980 20 add_watcher() i=1 2024-03-04T18:14:57.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.160+0000 7f45a8c48980 20 add_watcher() i=2 2024-03-04T18:14:57.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.160+0000 7f45a8c48980 20 add_watcher() i=3 2024-03-04T18:14:57.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.160+0000 7f45a8c48980 20 add_watcher() i=4 2024-03-04T18:14:57.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.160+0000 7f45a8c48980 20 add_watcher() i=5 2024-03-04T18:14:57.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.160+0000 7f45a8c48980 20 add_watcher() i=6 2024-03-04T18:14:57.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.164+0000 7f45a8c48980 20 add_watcher() i=7 2024-03-04T18:14:57.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.164+0000 7f45a8c48980 2 all 8 watchers are set, enabling cache 2024-03-04T18:14:57.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.168+0000 7f445cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:14:57.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.168+0000 7f45a8c48980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:14:57.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.168+0000 7f45a8c48980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:14:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.260+0000 7f43d77fe640 20 reqs_thread_entry: start 2024-03-04T18:14:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.260+0000 7f45a8c48980 20 init_complete bucket index max shards: 1 2024-03-04T18:14:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.260+0000 7f45a8c48980 20 Filter name: none 2024-03-04T18:14:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.260+0000 7f43d57fa640 20 reqs_thread_entry: start 2024-03-04T18:14:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.260+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:14:57.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.264+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.264+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.264+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.268+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:14:57.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.272+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.272+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.272+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:14:57.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.272+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.17370103053834081407', time=2024-03-04T20:11:41.748345+0000, chain.objs.size()=21 2024-03-04T18:14:57.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.272+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.1 2024-03-04T18:14:57.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.272+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.1_1 2024-03-04T18:14:57.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.272+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.2 2024-03-04T18:14:57.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.272+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.2_1 2024-03-04T18:14:57.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.272+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.3 2024-03-04T18:14:57.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.272+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.3_1 2024-03-04T18:14:57.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.276+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.4 2024-03-04T18:14:57.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.276+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.4_1 2024-03-04T18:14:57.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.276+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.5 2024-03-04T18:14:57.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.276+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.5_1 2024-03-04T18:14:57.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.276+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.6 2024-03-04T18:14:57.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.276+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.6_1 2024-03-04T18:14:57.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.280+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.7 2024-03-04T18:14:57.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.280+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.7_1 2024-03-04T18:14:57.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.280+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.8 2024-03-04T18:14:57.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.280+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.8_1 2024-03-04T18:14:57.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.280+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.9 2024-03-04T18:14:57.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.280+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.9_1 2024-03-04T18:14:57.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.280+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.10 2024-03-04T18:14:57.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.284+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.10_1 2024-03-04T18:14:57.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.284+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~PWB8gtq3p-aXRvx_eVzEtMUQpIRpM4J.11 2024-03-04T18:14:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.292+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.296+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:14:57.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.300+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.300+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.300+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.304+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:14:57.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.304+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.308+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.308+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:14:57.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.308+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.1629381061193697319', time=2024-03-04T20:09:09.042451+0000, chain.objs.size()=7 2024-03-04T18:14:57.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.308+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~RJIlie9a24i4oGtDiS9p9EItUJtiWX6.1 2024-03-04T18:14:57.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.308+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~RJIlie9a24i4oGtDiS9p9EItUJtiWX6.1_1 2024-03-04T18:14:57.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.308+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~RJIlie9a24i4oGtDiS9p9EItUJtiWX6.1_2 2024-03-04T18:14:57.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.308+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~RJIlie9a24i4oGtDiS9p9EItUJtiWX6.2 2024-03-04T18:14:57.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.308+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~RJIlie9a24i4oGtDiS9p9EItUJtiWX6.2_1 2024-03-04T18:14:57.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.308+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~RJIlie9a24i4oGtDiS9p9EItUJtiWX6.2_2 2024-03-04T18:14:57.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.308+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~RJIlie9a24i4oGtDiS9p9EItUJtiWX6.3 2024-03-04T18:14:57.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.316+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.320+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:14:57.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:14:57.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16127982269783924884', time=2024-03-04T20:12:30.208241+0000, chain.objs.size()=19 2024-03-04T18:14:57.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.1 2024-03-04T18:14:57.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.1_1 2024-03-04T18:14:57.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.2 2024-03-04T18:14:57.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.2_1 2024-03-04T18:14:57.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.3 2024-03-04T18:14:57.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.3_1 2024-03-04T18:14:57.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.4 2024-03-04T18:14:57.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.4_1 2024-03-04T18:14:57.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.5 2024-03-04T18:14:57.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.5_1 2024-03-04T18:14:57.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.6 2024-03-04T18:14:57.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.324+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.6_1 2024-03-04T18:14:57.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.332+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.7 2024-03-04T18:14:57.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.332+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.7_1 2024-03-04T18:14:57.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.332+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.8 2024-03-04T18:14:57.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.332+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.8_1 2024-03-04T18:14:57.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.336+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.9 2024-03-04T18:14:57.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.336+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.9_1 2024-03-04T18:14:57.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.336+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~tyRGj6LjnjmFrfeowB3Z7BYEQNtYbgX.10 2024-03-04T18:14:57.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.336+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4020190012809331283', time=2024-03-04T20:13:24.009132+0000, chain.objs.size()=16 2024-03-04T18:14:57.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.336+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.1 2024-03-04T18:14:57.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.340+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.1_1 2024-03-04T18:14:57.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.340+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.1_2 2024-03-04T18:14:57.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.340+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.2 2024-03-04T18:14:57.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.340+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.2_1 2024-03-04T18:14:57.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.344+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.2_2 2024-03-04T18:14:57.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.344+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.3 2024-03-04T18:14:57.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.344+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.3_1 2024-03-04T18:14:57.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.344+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.3_2 2024-03-04T18:14:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.344+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.4 2024-03-04T18:14:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.344+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.4_1 2024-03-04T18:14:57.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.344+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.4_2 2024-03-04T18:14:57.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.344+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.5 2024-03-04T18:14:57.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.344+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.5_1 2024-03-04T18:14:57.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.344+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.5_2 2024-03-04T18:14:57.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.348+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~2czKHbdMmAftS-9l_PK0HjfIx4PGl-c.6 2024-03-04T18:14:57.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.356+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.360+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:14:57.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.364+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.364+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.368+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:14:57.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.368+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8893892516425856628', time=2024-03-04T20:09:09.404633+0000, chain.objs.size()=5 2024-03-04T18:14:57.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.368+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.36__shadow_.XJgGp4CiVITIgrwXsFJXcg5QJ_ImgSf_1 2024-03-04T18:14:57.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.368+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.36__shadow_.XJgGp4CiVITIgrwXsFJXcg5QJ_ImgSf_2 2024-03-04T18:14:57.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.368+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.36__shadow_.XJgGp4CiVITIgrwXsFJXcg5QJ_ImgSf_3 2024-03-04T18:14:57.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.368+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.36__shadow_.XJgGp4CiVITIgrwXsFJXcg5QJ_ImgSf_4 2024-03-04T18:14:57.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.368+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.36__shadow_.XJgGp4CiVITIgrwXsFJXcg5QJ_ImgSf_5 2024-03-04T18:14:57.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.372+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.376+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:14:57.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.376+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:14:57.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12248908059420610240', time=2024-03-04T20:07:57.135266+0000, chain.objs.size()=9 2024-03-04T18:14:57.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.32__multipart_key1.2~5HlcVgeT-K5APZJcDkPjWqEajnH331M.1 2024-03-04T18:14:57.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.32__shadow_key1.2~5HlcVgeT-K5APZJcDkPjWqEajnH331M.1_1 2024-03-04T18:14:57.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.32__multipart_key1.2~5HlcVgeT-K5APZJcDkPjWqEajnH331M.2 2024-03-04T18:14:57.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.32__shadow_key1.2~5HlcVgeT-K5APZJcDkPjWqEajnH331M.2_1 2024-03-04T18:14:57.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.32__multipart_key1.2~5HlcVgeT-K5APZJcDkPjWqEajnH331M.3 2024-03-04T18:14:57.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.32__shadow_key1.2~5HlcVgeT-K5APZJcDkPjWqEajnH331M.3_1 2024-03-04T18:14:57.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.32__multipart_key1.2~5HlcVgeT-K5APZJcDkPjWqEajnH331M.4 2024-03-04T18:14:57.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.32__shadow_key1.2~5HlcVgeT-K5APZJcDkPjWqEajnH331M.4_1 2024-03-04T18:14:57.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.32__multipart_key1.2~5HlcVgeT-K5APZJcDkPjWqEajnH331M.5 2024-03-04T18:14:57.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13546733843427253308', time=2024-03-04T20:14:56.043532+0000, chain.objs.size()=21 2024-03-04T18:14:57.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__multipart_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.1 2024-03-04T18:14:57.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__shadow_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.1_1 2024-03-04T18:14:57.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.380+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__multipart_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.2 2024-03-04T18:14:57.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.388+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__shadow_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.2_1 2024-03-04T18:14:57.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.388+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__multipart_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.3 2024-03-04T18:14:57.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.388+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__shadow_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.3_1 2024-03-04T18:14:57.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.388+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__multipart_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.4 2024-03-04T18:14:57.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.388+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__shadow_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.4_1 2024-03-04T18:14:57.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.388+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__multipart_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.5 2024-03-04T18:14:57.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.388+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__shadow_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.5_1 2024-03-04T18:14:57.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.388+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__multipart_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.6 2024-03-04T18:14:57.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.388+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__shadow_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.6_1 2024-03-04T18:14:57.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.392+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__multipart_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.7 2024-03-04T18:14:57.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.392+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__shadow_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.7_1 2024-03-04T18:14:57.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.396+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__multipart_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.8 2024-03-04T18:14:57.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.396+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__shadow_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.8_1 2024-03-04T18:14:57.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.396+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__multipart_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.9 2024-03-04T18:14:57.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.396+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__shadow_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.9_1 2024-03-04T18:14:57.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.396+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__multipart_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.10 2024-03-04T18:14:57.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.396+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__shadow_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.10_1 2024-03-04T18:14:57.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.396+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.56__multipart_key1.2~HP1uM9xjS90k4Kg0TYXnDy8lvsRxoQd.11 2024-03-04T18:14:57.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.404+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.408+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:14:57.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.408+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.408+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.412+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:14:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.412+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5742313251912014143', time=2024-03-04T20:11:12.217386+0000, chain.objs.size()=21 2024-03-04T18:14:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.412+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.1 2024-03-04T18:14:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.412+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.1_1 2024-03-04T18:14:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.412+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.2 2024-03-04T18:14:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.412+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.2_1 2024-03-04T18:14:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.412+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.3 2024-03-04T18:14:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.412+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.3_1 2024-03-04T18:14:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.412+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.4 2024-03-04T18:14:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.412+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.4_1 2024-03-04T18:14:57.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.412+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.5 2024-03-04T18:14:57.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.412+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.5_1 2024-03-04T18:14:57.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.412+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.6 2024-03-04T18:14:57.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.412+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.6_1 2024-03-04T18:14:57.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.420+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.7 2024-03-04T18:14:57.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.420+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.7_1 2024-03-04T18:14:57.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.420+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.8 2024-03-04T18:14:57.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.420+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.8_1 2024-03-04T18:14:57.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.420+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.9 2024-03-04T18:14:57.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.420+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.9_1 2024-03-04T18:14:57.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.420+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.10 2024-03-04T18:14:57.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.420+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.10_1 2024-03-04T18:14:57.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.420+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Xrqo1M4etgW-ZoDtu4t7yXv59BQAlBU.11 2024-03-04T18:14:57.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.428+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.432+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:14:57.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.432+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.432+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.436+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.436+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:14:57.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.440+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.440+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.440+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.440+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:14:57.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.444+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.444+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.444+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:14:57.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.444+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11058587668528055848', time=2024-03-04T20:09:01.539143+0000, chain.objs.size()=9 2024-03-04T18:14:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.444+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~gz-x_R4XC59H4hJ6VfvIJfF5tbF0L36.1 2024-03-04T18:14:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.448+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~gz-x_R4XC59H4hJ6VfvIJfF5tbF0L36.1_1 2024-03-04T18:14:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.448+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~gz-x_R4XC59H4hJ6VfvIJfF5tbF0L36.2 2024-03-04T18:14:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.448+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~gz-x_R4XC59H4hJ6VfvIJfF5tbF0L36.2_1 2024-03-04T18:14:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.448+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~gz-x_R4XC59H4hJ6VfvIJfF5tbF0L36.3 2024-03-04T18:14:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.448+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~gz-x_R4XC59H4hJ6VfvIJfF5tbF0L36.3_1 2024-03-04T18:14:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.448+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~gz-x_R4XC59H4hJ6VfvIJfF5tbF0L36.4 2024-03-04T18:14:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.448+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~gz-x_R4XC59H4hJ6VfvIJfF5tbF0L36.4_1 2024-03-04T18:14:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.448+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~gz-x_R4XC59H4hJ6VfvIJfF5tbF0L36.5 2024-03-04T18:14:57.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.452+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.456+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:14:57.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.460+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.460+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.460+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.460+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:14:57.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.464+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:14:57.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.18288594978427192694', time=2024-03-04T20:12:56.974940+0000, chain.objs.size()=16 2024-03-04T18:14:57.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.1 2024-03-04T18:14:57.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.1_1 2024-03-04T18:14:57.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.1_2 2024-03-04T18:14:57.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.2 2024-03-04T18:14:57.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.2_1 2024-03-04T18:14:57.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.2_2 2024-03-04T18:14:57.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.3 2024-03-04T18:14:57.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.3_1 2024-03-04T18:14:57.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.3_2 2024-03-04T18:14:57.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.4 2024-03-04T18:14:57.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.4_1 2024-03-04T18:14:57.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.468+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.4_2 2024-03-04T18:14:57.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.472+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.5 2024-03-04T18:14:57.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.472+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.5_1 2024-03-04T18:14:57.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.472+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.5_2 2024-03-04T18:14:57.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.476+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~Oapb8drOfoBR0mW3zRmRebrCjz3GX7O.6 2024-03-04T18:14:57.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.480+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.484+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:14:57.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.488+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.488+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.488+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.488+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:14:57.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.492+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:14:57.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.14022866893508320807', time=2024-03-04T20:09:09.804952+0000, chain.objs.size()=7 2024-03-04T18:14:57.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~qiAmRnnxbjf0BrfkAL5HYGWprklqJFg.1 2024-03-04T18:14:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~qiAmRnnxbjf0BrfkAL5HYGWprklqJFg.1_1 2024-03-04T18:14:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~qiAmRnnxbjf0BrfkAL5HYGWprklqJFg.1_2 2024-03-04T18:14:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~qiAmRnnxbjf0BrfkAL5HYGWprklqJFg.2 2024-03-04T18:14:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~qiAmRnnxbjf0BrfkAL5HYGWprklqJFg.2_1 2024-03-04T18:14:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~qiAmRnnxbjf0BrfkAL5HYGWprklqJFg.2_2 2024-03-04T18:14:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~qiAmRnnxbjf0BrfkAL5HYGWprklqJFg.3 2024-03-04T18:14:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7579138368602253222', time=2024-03-04T20:13:41.372707+0000, chain.objs.size()=9 2024-03-04T18:14:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.46__multipart_key1.2~wgqSfljctWobCiwQayaPFbBQlk3MsgA.1 2024-03-04T18:14:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.46__shadow_key1.2~wgqSfljctWobCiwQayaPFbBQlk3MsgA.1_1 2024-03-04T18:14:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.46__multipart_key1.2~wgqSfljctWobCiwQayaPFbBQlk3MsgA.2 2024-03-04T18:14:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.46__shadow_key1.2~wgqSfljctWobCiwQayaPFbBQlk3MsgA.2_1 2024-03-04T18:14:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.496+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.46__multipart_key1.2~wgqSfljctWobCiwQayaPFbBQlk3MsgA.3 2024-03-04T18:14:57.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.500+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.46__shadow_key1.2~wgqSfljctWobCiwQayaPFbBQlk3MsgA.3_1 2024-03-04T18:14:57.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.500+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.46__multipart_key1.2~wgqSfljctWobCiwQayaPFbBQlk3MsgA.4 2024-03-04T18:14:57.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.500+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.46__shadow_key1.2~wgqSfljctWobCiwQayaPFbBQlk3MsgA.4_1 2024-03-04T18:14:57.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.500+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.46__multipart_key1.2~wgqSfljctWobCiwQayaPFbBQlk3MsgA.5 2024-03-04T18:14:57.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.508+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.512+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:14:57.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.512+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.512+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.512+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.516+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:14:57.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.516+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.520+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.520+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.520+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:14:57.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.524+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.524+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.524+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.528+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:14:57.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.528+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.532+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.532+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.532+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:14:57.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.536+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.536+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.536+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:14:57.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.536+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.16726265439846656307', time=2024-03-04T20:10:08.560746+0000, chain.objs.size()=9 2024-03-04T18:14:57.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.536+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.43__multipart_file.mpg.2~zmDknQKv2QrerkBbPJyBXwL0Z0lHfk6.1 2024-03-04T18:14:57.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.536+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.43__shadow_file.mpg.2~zmDknQKv2QrerkBbPJyBXwL0Z0lHfk6.1_1 2024-03-04T18:14:57.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.536+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.43__multipart_file.mpg.2~zmDknQKv2QrerkBbPJyBXwL0Z0lHfk6.2 2024-03-04T18:14:57.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.536+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.43__shadow_file.mpg.2~zmDknQKv2QrerkBbPJyBXwL0Z0lHfk6.2_1 2024-03-04T18:14:57.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.536+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.43__multipart_file.mpg.2~zmDknQKv2QrerkBbPJyBXwL0Z0lHfk6.3 2024-03-04T18:14:57.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.536+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.43__shadow_file.mpg.2~zmDknQKv2QrerkBbPJyBXwL0Z0lHfk6.3_1 2024-03-04T18:14:57.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.536+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.43__multipart_file.mpg.2~zmDknQKv2QrerkBbPJyBXwL0Z0lHfk6.4 2024-03-04T18:14:57.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.536+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.43__shadow_file.mpg.2~zmDknQKv2QrerkBbPJyBXwL0Z0lHfk6.4_1 2024-03-04T18:14:57.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.536+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.43__multipart_file.mpg.2~zmDknQKv2QrerkBbPJyBXwL0Z0lHfk6.5 2024-03-04T18:14:57.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.544+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.552+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:14:57.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.552+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.552+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.552+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.556+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:14:57.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-03-04T18:14:57.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2', time=2024-03-04T20:14:14.414687+0000, chain.objs.size()=1 2024-03-04T18:14:57.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.14 2024-03-04T18:14:57.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2', time=2024-03-04T20:14:14.502721+0000, chain.objs.size()=1 2024-03-04T18:14:57.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.15 2024-03-04T18:14:57.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2', time=2024-03-04T20:14:14.744880+0000, chain.objs.size()=1 2024-03-04T18:14:57.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.16 2024-03-04T18:14:57.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2', time=2024-03-04T20:14:14.835313+0000, chain.objs.size()=1 2024-03-04T18:14:57.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.19 2024-03-04T18:14:57.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2', time=2024-03-04T20:14:15.122405+0000, chain.objs.size()=1 2024-03-04T18:14:57.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.20 2024-03-04T18:14:57.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2', time=2024-03-04T20:14:15.270953+0000, chain.objs.size()=25 2024-03-04T18:14:57.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.1 2024-03-04T18:14:57.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.2 2024-03-04T18:14:57.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.3 2024-03-04T18:14:57.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.4 2024-03-04T18:14:57.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.5 2024-03-04T18:14:57.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.6 2024-03-04T18:14:57.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.560+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.7 2024-03-04T18:14:57.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.564+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.8 2024-03-04T18:14:57.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.564+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.9 2024-03-04T18:14:57.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.564+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.10 2024-03-04T18:14:57.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.564+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.11 2024-03-04T18:14:57.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.564+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.12 2024-03-04T18:14:57.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.564+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.13 2024-03-04T18:14:57.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.564+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.0lb6sTiTq3Lwplc2IviJa4N1TjfPDQ_4.14 2024-03-04T18:14:57.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.564+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.31e_px80HX4oiTyk8NdX9_3x-1L2E6nu.15 2024-03-04T18:14:57.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.564+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.ZNywHn321fF-QW4m0mwpnnwf2n61jM3A.16 2024-03-04T18:14:57.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.568+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.17 2024-03-04T18:14:57.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.568+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.18 2024-03-04T18:14:57.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.568+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.HESP5pcDYnHHMVUfY5kWxD7DMdvMfOeG.19 2024-03-04T18:14:57.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.568+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.xZGtVxtPtexLExRc_ywoXOm8c7N6pB6L.20 2024-03-04T18:14:57.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.568+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.21 2024-03-04T18:14:57.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.568+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.22 2024-03-04T18:14:57.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.568+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.23 2024-03-04T18:14:57.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.572+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.24 2024-03-04T18:14:57.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.572+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.48__multipart_key1.2~6BnrroUtQ8b-KdjKFYeTItRTjyYKJS2.25 2024-03-04T18:14:57.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.576+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.580+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:14:57.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.584+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.584+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.584+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.588+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:14:57.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.588+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.592+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.592+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:14:57.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.592+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.6294837922557024635', time=2024-03-04T20:08:12.436709+0000, chain.objs.size()=9 2024-03-04T18:14:57.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.592+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__multipart_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.1 2024-03-04T18:14:57.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.592+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__shadow_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.1_1 2024-03-04T18:14:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.592+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__multipart_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.2 2024-03-04T18:14:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.592+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__shadow_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.2_1 2024-03-04T18:14:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.592+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__multipart_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.3 2024-03-04T18:14:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.592+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__shadow_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.3_1 2024-03-04T18:14:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.592+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__multipart_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.4 2024-03-04T18:14:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.592+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__shadow_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.4_1 2024-03-04T18:14:57.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.592+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__multipart_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.5 2024-03-04T18:14:57.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.596+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.600+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:14:57.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.604+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.604+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.604+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.608+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:14:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.608+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.612+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.612+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:14:57.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.612+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5982131355632131836', time=2024-03-04T20:09:05.407426+0000, chain.objs.size()=7 2024-03-04T18:14:57.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.612+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~Bb6HTgLgwR5nEUrq5gjafyu0kWcg8s9.1 2024-03-04T18:14:57.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.612+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~Bb6HTgLgwR5nEUrq5gjafyu0kWcg8s9.1_1 2024-03-04T18:14:57.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.612+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~Bb6HTgLgwR5nEUrq5gjafyu0kWcg8s9.1_2 2024-03-04T18:14:57.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.612+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~Bb6HTgLgwR5nEUrq5gjafyu0kWcg8s9.2 2024-03-04T18:14:57.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.612+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~Bb6HTgLgwR5nEUrq5gjafyu0kWcg8s9.2_1 2024-03-04T18:14:57.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.612+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~Bb6HTgLgwR5nEUrq5gjafyu0kWcg8s9.2_2 2024-03-04T18:14:57.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.612+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~Bb6HTgLgwR5nEUrq5gjafyu0kWcg8s9.3 2024-03-04T18:14:57.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.620+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.628+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:14:57.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.628+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:14:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__shadow_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.1_1_030161b69872096063ebf2e81a0db400', time=2024-03-04T20:08:12.837242+0000, chain.objs.size()=9 2024-03-04T18:14:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__multipart_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.1 2024-03-04T18:14:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__shadow_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.1_1 2024-03-04T18:14:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__multipart_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.2 2024-03-04T18:14:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__shadow_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.2_1 2024-03-04T18:14:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__multipart_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.3 2024-03-04T18:14:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__shadow_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.3_1 2024-03-04T18:14:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__multipart_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.4 2024-03-04T18:14:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__shadow_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.4_1 2024-03-04T18:14:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.34__multipart_key1.2~X5qi5Y0qWZqG6aUX_NMjiyp6xSUWB_X.5 2024-03-04T18:14:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.421563244634109413', time=2024-03-04T20:13:24.341487+0000, chain.objs.size()=16 2024-03-04T18:14:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.1 2024-03-04T18:14:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.1_1 2024-03-04T18:14:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.632+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.1_2 2024-03-04T18:14:57.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.636+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.2 2024-03-04T18:14:57.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.636+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.2_1 2024-03-04T18:14:57.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.636+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.2_2 2024-03-04T18:14:57.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.636+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.3 2024-03-04T18:14:57.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.636+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.3_1 2024-03-04T18:14:57.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.636+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.3_2 2024-03-04T18:14:57.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.636+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.4 2024-03-04T18:14:57.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.636+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.4_1 2024-03-04T18:14:57.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.636+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.4_2 2024-03-04T18:14:57.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.636+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.5 2024-03-04T18:14:57.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.640+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.5_1 2024-03-04T18:14:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.644+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__shadow_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.5_2 2024-03-04T18:14:57.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.644+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.45__multipart_key1.2~j3VRI0oLxgDM-Pjuimv3Alj7pmzH5nj.6 2024-03-04T18:14:57.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.648+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.652+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:14:57.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.656+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.656+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.656+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:14:57.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.656+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8094632649543696652', time=2024-03-04T20:08:57.687542+0000, chain.objs.size()=9 2024-03-04T18:14:57.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.656+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~ziA1D0mkIuqpRCW_hAOJGmyUk64P09c.1 2024-03-04T18:14:57.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.656+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~ziA1D0mkIuqpRCW_hAOJGmyUk64P09c.1_1 2024-03-04T18:14:57.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.656+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~ziA1D0mkIuqpRCW_hAOJGmyUk64P09c.2 2024-03-04T18:14:57.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.656+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~ziA1D0mkIuqpRCW_hAOJGmyUk64P09c.2_1 2024-03-04T18:14:57.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.656+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~ziA1D0mkIuqpRCW_hAOJGmyUk64P09c.3 2024-03-04T18:14:57.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.656+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~ziA1D0mkIuqpRCW_hAOJGmyUk64P09c.3_1 2024-03-04T18:14:57.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.656+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~ziA1D0mkIuqpRCW_hAOJGmyUk64P09c.4 2024-03-04T18:14:57.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.656+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~ziA1D0mkIuqpRCW_hAOJGmyUk64P09c.4_1 2024-03-04T18:14:57.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.656+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~ziA1D0mkIuqpRCW_hAOJGmyUk64P09c.5 2024-03-04T18:14:57.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.664+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.668+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:14:57.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.672+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.672+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.676+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:14:57.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.676+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7790887327129669467', time=2024-03-04T20:08:43.535942+0000, chain.objs.size()=9 2024-03-04T18:14:57.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.676+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~HB0fmcwvY2wUOW24p-Cj_bEEgIQttLW.1 2024-03-04T18:14:57.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.676+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~HB0fmcwvY2wUOW24p-Cj_bEEgIQttLW.1_1 2024-03-04T18:14:57.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.676+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~HB0fmcwvY2wUOW24p-Cj_bEEgIQttLW.2 2024-03-04T18:14:57.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.676+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~HB0fmcwvY2wUOW24p-Cj_bEEgIQttLW.2_1 2024-03-04T18:14:57.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.676+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~HB0fmcwvY2wUOW24p-Cj_bEEgIQttLW.3 2024-03-04T18:14:57.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.676+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~HB0fmcwvY2wUOW24p-Cj_bEEgIQttLW.3_1 2024-03-04T18:14:57.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.676+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~HB0fmcwvY2wUOW24p-Cj_bEEgIQttLW.4 2024-03-04T18:14:57.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.676+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__shadow_key1.2~HB0fmcwvY2wUOW24p-Cj_bEEgIQttLW.4_1 2024-03-04T18:14:57.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.676+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.37__multipart_key1.2~HB0fmcwvY2wUOW24p-Cj_bEEgIQttLW.5 2024-03-04T18:14:57.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.684+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.688+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:14:57.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-03-04T18:14:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.972409703804995447', time=2024-03-04T20:09:24.887014+0000, chain.objs.size()=9 2024-03-04T18:14:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.41__multipart_key1.2~tV6LzjQRXRwmsqinSLCtg2Tn2_2U52P.1 2024-03-04T18:14:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.41__shadow_key1.2~tV6LzjQRXRwmsqinSLCtg2Tn2_2U52P.1_1 2024-03-04T18:14:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.41__multipart_key1.2~tV6LzjQRXRwmsqinSLCtg2Tn2_2U52P.2 2024-03-04T18:14:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.41__shadow_key1.2~tV6LzjQRXRwmsqinSLCtg2Tn2_2U52P.2_1 2024-03-04T18:14:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.41__multipart_key1.2~tV6LzjQRXRwmsqinSLCtg2Tn2_2U52P.3 2024-03-04T18:14:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.41__shadow_key1.2~tV6LzjQRXRwmsqinSLCtg2Tn2_2U52P.3_1 2024-03-04T18:14:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.41__multipart_key1.2~tV6LzjQRXRwmsqinSLCtg2Tn2_2U52P.4 2024-03-04T18:14:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.41__shadow_key1.2~tV6LzjQRXRwmsqinSLCtg2Tn2_2U52P.4_1 2024-03-04T18:14:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.41__multipart_key1.2~tV6LzjQRXRwmsqinSLCtg2Tn2_2U52P.5 2024-03-04T18:14:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13024293745002718759', time=2024-03-04T20:09:42.086199+0000, chain.objs.size()=9 2024-03-04T18:14:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.42__multipart_key1.2~t8QYRX08VfKp67TR2npEkdnnLIxJT0-.1 2024-03-04T18:14:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.42__shadow_key1.2~t8QYRX08VfKp67TR2npEkdnnLIxJT0-.1_1 2024-03-04T18:14:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.692+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.42__multipart_key1.2~t8QYRX08VfKp67TR2npEkdnnLIxJT0-.2 2024-03-04T18:14:57.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.704+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.42__shadow_key1.2~t8QYRX08VfKp67TR2npEkdnnLIxJT0-.2_1 2024-03-04T18:14:57.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.704+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.42__multipart_key1.2~t8QYRX08VfKp67TR2npEkdnnLIxJT0-.3 2024-03-04T18:14:57.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.704+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.42__shadow_key1.2~t8QYRX08VfKp67TR2npEkdnnLIxJT0-.3_1 2024-03-04T18:14:57.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.704+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.42__multipart_key1.2~t8QYRX08VfKp67TR2npEkdnnLIxJT0-.4 2024-03-04T18:14:57.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.704+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.42__shadow_key1.2~t8QYRX08VfKp67TR2npEkdnnLIxJT0-.4_1 2024-03-04T18:14:57.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.704+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.42__multipart_key1.2~t8QYRX08VfKp67TR2npEkdnnLIxJT0-.5 2024-03-04T18:14:57.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.704+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5431833022329385875', time=2024-03-04T20:14:03.736609+0000, chain.objs.size()=9 2024-03-04T18:14:57.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.704+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.47__multipart_key1.2~wY0EGRxLsagYQjf5QedoYv7mgCOhePv.1 2024-03-04T18:14:57.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.704+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.47__shadow_key1.2~wY0EGRxLsagYQjf5QedoYv7mgCOhePv.1_1 2024-03-04T18:14:57.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.704+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.47__multipart_key1.2~wY0EGRxLsagYQjf5QedoYv7mgCOhePv.2 2024-03-04T18:14:57.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.704+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.47__shadow_key1.2~wY0EGRxLsagYQjf5QedoYv7mgCOhePv.2_1 2024-03-04T18:14:57.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.704+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.47__multipart_key1.2~wY0EGRxLsagYQjf5QedoYv7mgCOhePv.3 2024-03-04T18:14:57.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.708+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.47__shadow_key1.2~wY0EGRxLsagYQjf5QedoYv7mgCOhePv.3_1 2024-03-04T18:14:57.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.712+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.47__multipart_key1.2~wY0EGRxLsagYQjf5QedoYv7mgCOhePv.4 2024-03-04T18:14:57.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.712+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.47__shadow_key1.2~wY0EGRxLsagYQjf5QedoYv7mgCOhePv.4_1 2024-03-04T18:14:57.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.712+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.47__multipart_key1.2~wY0EGRxLsagYQjf5QedoYv7mgCOhePv.5 2024-03-04T18:14:57.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.720+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.724+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:14:57.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.724+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.728+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.728+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:14:57.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.728+0000 7f45a8c48980 20 garbage collection: RGWGC::process iterating over entry tag='2~O8NnHEu7ibaImn78lYMUArS2cjv6k9D', time=2024-03-04T20:07:34.060996+0000, chain.objs.size()=9 2024-03-04T18:14:57.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.728+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.30__multipart_key1.2~O8NnHEu7ibaImn78lYMUArS2cjv6k9D.1 2024-03-04T18:14:57.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.728+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.30__shadow_key1.2~O8NnHEu7ibaImn78lYMUArS2cjv6k9D.1_1 2024-03-04T18:14:57.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.728+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.30__multipart_key1.2~O8NnHEu7ibaImn78lYMUArS2cjv6k9D.2 2024-03-04T18:14:57.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.728+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.30__shadow_key1.2~O8NnHEu7ibaImn78lYMUArS2cjv6k9D.2_1 2024-03-04T18:14:57.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.728+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.30__multipart_key1.2~O8NnHEu7ibaImn78lYMUArS2cjv6k9D.3 2024-03-04T18:14:57.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.728+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.30__shadow_key1.2~O8NnHEu7ibaImn78lYMUArS2cjv6k9D.3_1 2024-03-04T18:14:57.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.728+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.30__multipart_key1.2~O8NnHEu7ibaImn78lYMUArS2cjv6k9D.4 2024-03-04T18:14:57.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.728+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.30__shadow_key1.2~O8NnHEu7ibaImn78lYMUArS2cjv6k9D.4_1 2024-03-04T18:14:57.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.728+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.30__multipart_key1.2~O8NnHEu7ibaImn78lYMUArS2cjv6k9D.5 2024-03-04T18:14:57.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.736+0000 7f45a8c48980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:14:57.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.740+0000 7f45a8c48980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:14:57.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.740+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.740+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:57.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.740+0000 7f45a8c48980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:57.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.744+0000 7f445cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:14:57.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.748+0000 7f45a8c48980 20 remove_watcher() i=0 2024-03-04T18:14:57.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.748+0000 7f45a8c48980 2 removed watcher, disabling cache 2024-03-04T18:14:57.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.752+0000 7f45a8c48980 20 remove_watcher() i=1 2024-03-04T18:14:57.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.752+0000 7f45a8c48980 20 remove_watcher() i=2 2024-03-04T18:14:57.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.756+0000 7f45a8c48980 20 remove_watcher() i=3 2024-03-04T18:14:57.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.756+0000 7f45a8c48980 20 remove_watcher() i=4 2024-03-04T18:14:57.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.760+0000 7f45a8c48980 20 remove_watcher() i=5 2024-03-04T18:14:57.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.760+0000 7f45a8c48980 20 remove_watcher() i=6 2024-03-04T18:14:57.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.764+0000 7f45a8c48980 20 remove_watcher() i=7 2024-03-04T18:14:57.785 DEBUG:teuthology.orchestra.run.smithi137:> radosgw-admin gc process --include-all 2024-03-04T18:14:57.855 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:14:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:14:57.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.896+0000 7f09e3092980 20 rados->read ofs=0 len=0 2024-03-04T18:14:57.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.900+0000 7f09e3092980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:57.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.900+0000 7f09e3092980 20 realm 2024-03-04T18:14:57.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.900+0000 7f09e3092980 20 rados->read ofs=0 len=0 2024-03-04T18:14:57.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.900+0000 7f09e3092980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:57.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.900+0000 7f09e3092980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:14:57.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.900+0000 7f09e3092980 20 rados->read ofs=0 len=0 2024-03-04T18:14:57.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.900+0000 7f09e3092980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:57.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.900+0000 7f09e3092980 20 rados->read ofs=0 len=0 2024-03-04T18:14:57.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.900+0000 7f09e3092980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:14:57.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.900+0000 7f09e3092980 20 rados->read ofs=0 len=0 2024-03-04T18:14:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.900+0000 7f09e3092980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:14:57.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.900+0000 7f09e3092980 20 searching for the correct realm 2024-03-04T18:14:57.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:57.908+0000 7f09e13ab640 0 -- 172.21.15.137:0/1802009912 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f09d802c910 msgr2=0x7f09d80103e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:14:58.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.124+0000 7f09e13ab640 0 -- 172.21.15.137:0/1802009912 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x557340736390 msgr2=0x557340738850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:14:58.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:14:58.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:14:58.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:14:58.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:14:58.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:14:58.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:14:58.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:58.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:14:58.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:14:58.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 zone default found 2024-03-04T18:14:58.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 4 Realm: () 2024-03-04T18:14:58.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:14:58.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:14:58.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:14:58.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 zonegroup default 2024-03-04T18:14:58.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:58.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.136+0000 7f09e3092980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.140+0000 7f09e3092980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:58.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.140+0000 7f09e3092980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.140+0000 7f09e3092980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:58.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.140+0000 7f09e3092980 20 started sync module instance, tier type = 2024-03-04T18:14:58.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.140+0000 7f09e3092980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:14:58.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.164+0000 7f09e3092980 20 add_watcher() i=0 2024-03-04T18:14:58.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.164+0000 7f09e3092980 20 add_watcher() i=1 2024-03-04T18:14:58.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.164+0000 7f09e3092980 20 add_watcher() i=2 2024-03-04T18:14:58.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.164+0000 7f09e3092980 20 add_watcher() i=3 2024-03-04T18:14:58.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.164+0000 7f09e3092980 20 add_watcher() i=4 2024-03-04T18:14:58.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.164+0000 7f09e3092980 20 add_watcher() i=5 2024-03-04T18:14:58.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.164+0000 7f09e3092980 20 add_watcher() i=6 2024-03-04T18:14:58.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.168+0000 7f09e3092980 20 add_watcher() i=7 2024-03-04T18:14:58.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.168+0000 7f09e3092980 2 all 8 watchers are set, enabling cache 2024-03-04T18:14:58.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.172+0000 7f0896fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:14:58.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.172+0000 7f09e3092980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:14:58.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.172+0000 7f09e3092980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:14:58.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.260+0000 7f0895fe3640 20 reqs_thread_entry: start 2024-03-04T18:14:58.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.260+0000 7f09e3092980 20 init_complete bucket index max shards: 1 2024-03-04T18:14:58.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.260+0000 7f09e3092980 20 Filter name: none 2024-03-04T18:14:58.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.260+0000 7f08077fe640 20 reqs_thread_entry: start 2024-03-04T18:14:58.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.260+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:14:58.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.264+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.264+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.268+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.268+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:14:58.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.272+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.272+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.276+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.276+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:14:58.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.280+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.280+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.284+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.284+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:14:58.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.288+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.288+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.292+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.292+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:14:58.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.296+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.296+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.300+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.300+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:14:58.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.304+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.304+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.304+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.308+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:14:58.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.312+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.312+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.312+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.316+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:14:58.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.320+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.320+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.320+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.324+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:14:58.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.328+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.328+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.328+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.332+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:14:58.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.336+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.336+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.336+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.336+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:14:58.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.340+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.344+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.344+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.348+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:14:58.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.352+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.352+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.352+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.356+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:14:58.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.360+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.360+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.360+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.364+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:14:58.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.368+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.368+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.368+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.372+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:14:58.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.376+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.376+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.376+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.380+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:14:58.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.380+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.384+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.384+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.384+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:14:58.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.388+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.392+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.392+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.392+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:14:58.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.396+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.396+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.396+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.400+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:14:58.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.404+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.404+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.404+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.404+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:14:58.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.408+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.408+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.408+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.412+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:14:58.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.416+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.416+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.416+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.420+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:14:58.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.420+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.424+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.424+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.424+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:14:58.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.428+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.428+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.432+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.432+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:14:58.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.436+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.440+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.440+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.444+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:14:58.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.448+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.452+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.452+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.456+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:14:58.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.456+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.460+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.460+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.460+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:14:58.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.464+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.468+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.468+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.472+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:14:58.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.476+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.476+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.476+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.480+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:14:58.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.484+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.484+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.488+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.488+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:14:58.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.492+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.492+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.496+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.496+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:14:58.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.500+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.500+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.504+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.504+0000 7f09e3092980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:14:58.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.508+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.512+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:58.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.512+0000 7f09e3092980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:58.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.516+0000 7f0896fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:14:58.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.520+0000 7f09e3092980 20 remove_watcher() i=0 2024-03-04T18:14:58.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.520+0000 7f09e3092980 2 removed watcher, disabling cache 2024-03-04T18:14:58.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.524+0000 7f09e3092980 20 remove_watcher() i=1 2024-03-04T18:14:58.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.524+0000 7f09e3092980 20 remove_watcher() i=2 2024-03-04T18:14:58.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.528+0000 7f09e3092980 20 remove_watcher() i=3 2024-03-04T18:14:58.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.532+0000 7f09e3092980 20 remove_watcher() i=4 2024-03-04T18:14:58.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.532+0000 7f09e3092980 20 remove_watcher() i=5 2024-03-04T18:14:58.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.536+0000 7f09e3092980 20 remove_watcher() i=6 2024-03-04T18:14:58.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.540+0000 7f09e3092980 20 remove_watcher() i=7 2024-03-04T18:14:58.560 DEBUG:teuthology.orchestra.run.smithi137:> radosgw-admin gc process --include-all 2024-03-04T18:14:58.631 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:14:58.631 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:14:58.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.668+0000 7fc5ceb1d980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.672+0000 7fc5ceb1d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:58.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.672+0000 7fc5ceb1d980 20 realm 2024-03-04T18:14:58.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.672+0000 7fc5ceb1d980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.672+0000 7fc5ceb1d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:58.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.672+0000 7fc5ceb1d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:14:58.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.672+0000 7fc5ceb1d980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.672+0000 7fc5ceb1d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:58.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.672+0000 7fc5ceb1d980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.672+0000 7fc5ceb1d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:14:58.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.676+0000 7fc5ceb1d980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.676+0000 7fc5ceb1d980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:14:58.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.676+0000 7fc5ceb1d980 20 searching for the correct realm 2024-03-04T18:14:58.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.680+0000 7fc5cce36640 0 -- 172.21.15.137:0/1127554847 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fc5c40b8440 msgr2=0x7fc5c40b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:14:58.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.896+0000 7fc5cce36640 0 -- 172.21.15.137:0/1127554847 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55ef5b05b4e0 msgr2=0x55ef5b05d9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:14:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.904+0000 7fc5ceb1d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:14:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.904+0000 7fc5ceb1d980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:14:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.904+0000 7fc5ceb1d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:14:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.904+0000 7fc5ceb1d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:14:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.904+0000 7fc5ceb1d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:14:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.904+0000 7fc5ceb1d980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:14:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.904+0000 7fc5ceb1d980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.904+0000 7fc5ceb1d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.904+0000 7fc5ceb1d980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.904+0000 7fc5ceb1d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:14:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.904+0000 7fc5ceb1d980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:14:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 20 zone default found 2024-03-04T18:14:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 4 Realm: () 2024-03-04T18:14:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:14:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:14:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:14:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 20 zonegroup default 2024-03-04T18:14:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 20 rados->read ofs=0 len=0 2024-03-04T18:14:58.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:58.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 20 started sync module instance, tier type = 2024-03-04T18:14:58.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.908+0000 7fc5ceb1d980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:14:58.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.932+0000 7fc5ceb1d980 20 add_watcher() i=0 2024-03-04T18:14:58.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.932+0000 7fc5ceb1d980 20 add_watcher() i=1 2024-03-04T18:14:58.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.932+0000 7fc5ceb1d980 20 add_watcher() i=2 2024-03-04T18:14:58.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.932+0000 7fc5ceb1d980 20 add_watcher() i=3 2024-03-04T18:14:58.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.932+0000 7fc5ceb1d980 20 add_watcher() i=4 2024-03-04T18:14:58.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.932+0000 7fc5ceb1d980 20 add_watcher() i=5 2024-03-04T18:14:58.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.932+0000 7fc5ceb1d980 20 add_watcher() i=6 2024-03-04T18:14:58.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.932+0000 7fc5ceb1d980 20 add_watcher() i=7 2024-03-04T18:14:58.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.932+0000 7fc5ceb1d980 2 all 8 watchers are set, enabling cache 2024-03-04T18:14:58.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.936+0000 7fc482fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:14:58.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.940+0000 7fc5ceb1d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:14:58.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:58.940+0000 7fc5ceb1d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:14:59.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.032+0000 7fc481fdb640 20 reqs_thread_entry: start 2024-03-04T18:14:59.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.032+0000 7fc5ceb1d980 20 init_complete bucket index max shards: 1 2024-03-04T18:14:59.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.032+0000 7fc5ceb1d980 20 Filter name: none 2024-03-04T18:14:59.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.032+0000 7fc3f37fe640 20 reqs_thread_entry: start 2024-03-04T18:14:59.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.032+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:14:59.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.032+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.036+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.036+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.036+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:14:59.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.040+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.040+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.040+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.044+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:14:59.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.044+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.048+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.048+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.052+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:14:59.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.056+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.056+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.056+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.060+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:14:59.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.064+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.064+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.064+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.068+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:14:59.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.072+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.072+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.076+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.076+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:14:59.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.080+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.080+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.084+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.084+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:14:59.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.088+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.088+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.088+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.092+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:14:59.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.096+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.096+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.096+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.100+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:14:59.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.100+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.104+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.104+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.104+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:14:59.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.108+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.112+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.112+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.116+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:14:59.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.120+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.120+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.120+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.124+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:14:59.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.128+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.128+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.128+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.132+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:14:59.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.136+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.136+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.136+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.140+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:14:59.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.144+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.148+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.148+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.148+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:14:59.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.152+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.156+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.156+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.160+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:14:59.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.164+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.164+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.164+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.168+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:14:59.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.168+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.172+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.172+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.176+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:14:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.176+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.180+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.180+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.184+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:14:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.188+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.188+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.188+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.192+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:14:59.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.196+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.196+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.196+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.200+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:14:59.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.204+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.204+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.204+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.208+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:14:59.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.208+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.212+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.212+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.212+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:14:59.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.216+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.220+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.220+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.220+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:14:59.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.224+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.228+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.228+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.228+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:14:59.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.232+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.232+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.236+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.236+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:14:59.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.240+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.244+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.244+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.248+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:14:59.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.248+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.252+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.252+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.256+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:14:59.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.256+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.260+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.260+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.260+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:14:59.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.264+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.268+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.268+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.268+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:14:59.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.272+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.276+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.276+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.280+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:14:59.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.280+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.284+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.284+0000 7fc5ceb1d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.288+0000 7fc482fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:14:59.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.292+0000 7fc5ceb1d980 20 remove_watcher() i=0 2024-03-04T18:14:59.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.292+0000 7fc5ceb1d980 2 removed watcher, disabling cache 2024-03-04T18:14:59.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.296+0000 7fc5ceb1d980 20 remove_watcher() i=1 2024-03-04T18:14:59.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.300+0000 7fc5ceb1d980 20 remove_watcher() i=2 2024-03-04T18:14:59.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.300+0000 7fc5ceb1d980 20 remove_watcher() i=3 2024-03-04T18:14:59.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.304+0000 7fc5ceb1d980 20 remove_watcher() i=4 2024-03-04T18:14:59.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.304+0000 7fc5ceb1d980 20 remove_watcher() i=5 2024-03-04T18:14:59.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.308+0000 7fc5ceb1d980 20 remove_watcher() i=6 2024-03-04T18:14:59.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.312+0000 7fc5ceb1d980 20 remove_watcher() i=7 2024-03-04T18:14:59.337 DEBUG:teuthology.orchestra.run.smithi137:> radosgw-admin gc process --include-all 2024-03-04T18:14:59.408 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:14:59.408 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:14:59.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.448+0000 7fe3833c4980 20 rados->read ofs=0 len=0 2024-03-04T18:14:59.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.452+0000 7fe3833c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:59.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.452+0000 7fe3833c4980 20 realm 2024-03-04T18:14:59.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.452+0000 7fe3833c4980 20 rados->read ofs=0 len=0 2024-03-04T18:14:59.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.452+0000 7fe3833c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:59.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.452+0000 7fe3833c4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:14:59.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.452+0000 7fe3833c4980 20 rados->read ofs=0 len=0 2024-03-04T18:14:59.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.452+0000 7fe3833c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:59.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.452+0000 7fe3833c4980 20 rados->read ofs=0 len=0 2024-03-04T18:14:59.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.452+0000 7fe3833c4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:14:59.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.452+0000 7fe3833c4980 20 rados->read ofs=0 len=0 2024-03-04T18:14:59.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.456+0000 7fe3833c4980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:14:59.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.456+0000 7fe3833c4980 20 searching for the correct realm 2024-03-04T18:14:59.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.460+0000 7fe3816dd640 0 -- 172.21.15.137:0/3106261181 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fe3700103e0 msgr2=0x7fe370010840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:14:59.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.676+0000 7fe3816dd640 0 -- 172.21.15.137:0/3106261181 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55b46a5f5de0 msgr2=0x55b46a5f82a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:14:59.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:14:59.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:14:59.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:14:59.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:14:59.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:14:59.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:14:59.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 20 rados->read ofs=0 len=0 2024-03-04T18:14:59.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:59.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 20 rados->read ofs=0 len=0 2024-03-04T18:14:59.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:14:59.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 20 rados->read ofs=0 len=0 2024-03-04T18:14:59.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:14:59.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 20 zone default found 2024-03-04T18:14:59.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 4 Realm: () 2024-03-04T18:14:59.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:14:59.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:14:59.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:14:59.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.684+0000 7fe3833c4980 20 zonegroup default 2024-03-04T18:14:59.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.688+0000 7fe3833c4980 20 rados->read ofs=0 len=0 2024-03-04T18:14:59.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.688+0000 7fe3833c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:59.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.688+0000 7fe3833c4980 20 rados->read ofs=0 len=0 2024-03-04T18:14:59.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.688+0000 7fe3833c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:59.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.688+0000 7fe3833c4980 20 rados->read ofs=0 len=0 2024-03-04T18:14:59.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.688+0000 7fe3833c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:14:59.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.688+0000 7fe3833c4980 20 started sync module instance, tier type = 2024-03-04T18:14:59.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.688+0000 7fe3833c4980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:14:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.712+0000 7fe3833c4980 20 add_watcher() i=0 2024-03-04T18:14:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.712+0000 7fe3833c4980 20 add_watcher() i=1 2024-03-04T18:14:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.712+0000 7fe3833c4980 20 add_watcher() i=2 2024-03-04T18:14:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.712+0000 7fe3833c4980 20 add_watcher() i=3 2024-03-04T18:14:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.712+0000 7fe3833c4980 20 add_watcher() i=4 2024-03-04T18:14:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.712+0000 7fe3833c4980 20 add_watcher() i=5 2024-03-04T18:14:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.712+0000 7fe3833c4980 20 add_watcher() i=6 2024-03-04T18:14:59.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.716+0000 7fe3833c4980 20 add_watcher() i=7 2024-03-04T18:14:59.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.716+0000 7fe3833c4980 2 all 8 watchers are set, enabling cache 2024-03-04T18:14:59.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.720+0000 7fe236fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:14:59.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.720+0000 7fe3833c4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:14:59.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.720+0000 7fe3833c4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:14:59.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.808+0000 7fe235fe3640 20 reqs_thread_entry: start 2024-03-04T18:14:59.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.808+0000 7fe3833c4980 20 init_complete bucket index max shards: 1 2024-03-04T18:14:59.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.808+0000 7fe3833c4980 20 Filter name: none 2024-03-04T18:14:59.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.808+0000 7fe1a77fe640 20 reqs_thread_entry: start 2024-03-04T18:14:59.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.808+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:14:59.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.812+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.812+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.812+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.816+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:14:59.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.816+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.820+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.820+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.820+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:14:59.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.824+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.824+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.828+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.828+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:14:59.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.832+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.832+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.836+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.836+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:14:59.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.840+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.840+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.840+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.844+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:14:59.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.848+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.848+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.848+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.852+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:14:59.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.852+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.852+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.852+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.856+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:14:59.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.860+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.860+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.860+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.864+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:14:59.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.864+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.868+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.868+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.868+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:14:59.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.872+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.876+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.876+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.880+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:14:59.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.880+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.884+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.884+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.888+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:14:59.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.888+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.892+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.892+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.892+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:14:59.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.896+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.900+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.900+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.904+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:14:59.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.904+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.908+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.908+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.912+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:14:59.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.916+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.916+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.920+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.920+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:14:59.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.924+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.928+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.928+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.928+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:14:59.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.932+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.936+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.936+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.940+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:14:59.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.940+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.944+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.944+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.948+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:14:59.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.952+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.956+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.956+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.960+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:14:59.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.964+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.964+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.964+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.968+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:14:59.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.972+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.972+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.976+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.976+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:14:59.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.980+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.984+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.984+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.988+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:14:59.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.992+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:14:59.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.992+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:14:59.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.992+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:14:59.996+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:15:00.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.000+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.004+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:00.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.004+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.004+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:15:00.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.008+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.012+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:00.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.012+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.012+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:15:00.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.016+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.016+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:00.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.016+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.020+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:15:00.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.024+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.024+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:00.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.024+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.028+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:15:00.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.032+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.032+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:00.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.032+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.036+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:15:00.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.040+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.040+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:00.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.040+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.044+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:15:00.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.048+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.048+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:00.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.048+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.052+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:15:00.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.056+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.056+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:00.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.056+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.060+0000 7fe3833c4980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:15:00.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.064+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.064+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:00.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.064+0000 7fe3833c4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:00.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.068+0000 7fe236fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:15:00.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.076+0000 7fe3833c4980 20 remove_watcher() i=0 2024-03-04T18:15:00.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.076+0000 7fe3833c4980 2 removed watcher, disabling cache 2024-03-04T18:15:00.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.076+0000 7fe3833c4980 20 remove_watcher() i=1 2024-03-04T18:15:00.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.080+0000 7fe3833c4980 20 remove_watcher() i=2 2024-03-04T18:15:00.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.080+0000 7fe3833c4980 20 remove_watcher() i=3 2024-03-04T18:15:00.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.084+0000 7fe3833c4980 20 remove_watcher() i=4 2024-03-04T18:15:00.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.084+0000 7fe3833c4980 20 remove_watcher() i=5 2024-03-04T18:15:00.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.088+0000 7fe3833c4980 20 remove_watcher() i=6 2024-03-04T18:15:00.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:00.092+0000 7fe3833c4980 20 remove_watcher() i=7 2024-03-04T18:15:00.115 DEBUG:teuthology.orchestra.run.smithi137:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests BucketTest 2024-03-04T18:15:00.816 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :clean 2024-03-04T18:15:01.713 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :compileJava 2024-03-04T18:15:01.713 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :processResources 2024-03-04T18:15:01.713 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :classes 2024-03-04T18:15:02.113 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:02.113 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :compileTestJava 2024-03-04T18:15:02.113 INFO:teuthology.orchestra.run.smithi137.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-03-04T18:15:02.113 INFO:teuthology.orchestra.run.smithi137.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-03-04T18:15:02.114 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:02.114 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :processTestResources NO-SOURCE 2024-03-04T18:15:02.114 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :testClasses 2024-03-04T18:15:04.113 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:04.113 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :test 2024-03-04T18:15:04.113 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:04.113 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests STANDARD_OUT 2024-03-04T18:15:04.113 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04 18:15:03,528 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-03-04T18:15:04.113 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:04.114 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty STARTED 2024-03-04T18:15:04.114 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:04.114 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty PASSED 2024-03-04T18:15:04.413 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:04.413 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch STARTED 2024-03-04T18:15:04.513 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:04.513 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch PASSED 2024-03-04T18:15:05.013 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:05.013 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable STARTED 2024-03-04T18:15:05.013 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:05.013 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable PASSED 2024-03-04T18:15:05.513 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:05.513 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete STARTED 2024-03-04T18:15:05.812 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:05.813 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete PASSED 2024-03-04T18:15:06.013 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:06.013 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty STARTED 2024-03-04T18:15:06.213 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:06.213 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty PASSED 2024-03-04T18:15:06.813 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:06.813 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist STARTED 2024-03-04T18:15:06.813 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:06.813 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist PASSED 2024-03-04T18:15:07.013 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:07.013 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct STARTED 2024-03-04T18:15:07.313 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:07.313 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct PASSED 2024-03-04T18:15:08.213 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:08.213 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty STARTED 2024-03-04T18:15:08.413 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:08.413 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty PASSED 2024-03-04T18:15:08.813 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:08.813 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketNotExist STARTED 2024-03-04T18:15:08.913 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:08.913 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Bucket tests > BucketTest.testBucketNotExist PASSED 2024-03-04T18:15:09.055 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:09.055 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------- 2024-03-04T18:15:09.055 INFO:teuthology.orchestra.run.smithi137.stdout:| Results: SUCCESS (9 tests, 9 successes, 0 failures, 0 skipped) | 2024-03-04T18:15:09.055 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------- 2024-03-04T18:15:09.056 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:09.056 INFO:teuthology.orchestra.run.smithi137.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-03-04T18:15:09.056 INFO:teuthology.orchestra.run.smithi137.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-03-04T18:15:09.056 INFO:teuthology.orchestra.run.smithi137.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-03-04T18:15:09.056 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:09.056 INFO:teuthology.orchestra.run.smithi137.stdout:BUILD SUCCESSFUL in 8s 2024-03-04T18:15:09.056 INFO:teuthology.orchestra.run.smithi137.stdout:5 actionable tasks: 5 executed 2024-03-04T18:15:09.063 DEBUG:teuthology.orchestra.run.smithi137:> radosgw-admin gc process --include-all 2024-03-04T18:15:09.088 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:15:09.088 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:15:09.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.124+0000 7f85d8119980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.128+0000 7f85d8119980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:09.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.128+0000 7f85d8119980 20 realm 2024-03-04T18:15:09.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.128+0000 7f85d8119980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.128+0000 7f85d8119980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:09.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.128+0000 7f85d8119980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:15:09.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.128+0000 7f85d8119980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.128+0000 7f85d8119980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:09.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.128+0000 7f85d8119980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.128+0000 7f85d8119980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:15:09.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.128+0000 7f85d8119980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.132+0000 7f85d8119980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:15:09.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.132+0000 7f85d8119980 20 searching for the correct realm 2024-03-04T18:15:09.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.136+0000 7f85d6432640 0 -- 172.21.15.137:0/2772518149 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f85c8059310 msgr2=0x7f85c8059a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:15:09.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.352+0000 7f85d6432640 0 -- 172.21.15.137:0/2772518149 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x5630d5158c60 msgr2=0x5630d515b0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:15:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.360+0000 7f85d8119980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:15:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.360+0000 7f85d8119980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:15:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.360+0000 7f85d8119980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:15:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.360+0000 7f85d8119980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:15:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.360+0000 7f85d8119980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:15:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.360+0000 7f85d8119980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:15:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.360+0000 7f85d8119980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.360+0000 7f85d8119980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:09.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.360+0000 7f85d8119980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.360+0000 7f85d8119980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:15:09.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.360+0000 7f85d8119980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:15:09.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 20 zone default found 2024-03-04T18:15:09.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 4 Realm: () 2024-03-04T18:15:09.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:15:09.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:15:09.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:15:09.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 20 zonegroup default 2024-03-04T18:15:09.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:09.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:09.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:09.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 20 started sync module instance, tier type = 2024-03-04T18:15:09.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.364+0000 7f85d8119980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:15:09.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.392+0000 7f85d8119980 20 add_watcher() i=0 2024-03-04T18:15:09.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.392+0000 7f85d8119980 20 add_watcher() i=1 2024-03-04T18:15:09.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.392+0000 7f85d8119980 20 add_watcher() i=2 2024-03-04T18:15:09.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.392+0000 7f85d8119980 20 add_watcher() i=3 2024-03-04T18:15:09.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.392+0000 7f85d8119980 20 add_watcher() i=4 2024-03-04T18:15:09.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.392+0000 7f85d8119980 20 add_watcher() i=5 2024-03-04T18:15:09.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.392+0000 7f85d8119980 20 add_watcher() i=6 2024-03-04T18:15:09.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.392+0000 7f85d8119980 20 add_watcher() i=7 2024-03-04T18:15:09.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.392+0000 7f85d8119980 2 all 8 watchers are set, enabling cache 2024-03-04T18:15:09.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.396+0000 7f848bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:15:09.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.400+0000 7f85d8119980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:15:09.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.400+0000 7f85d8119980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:15:09.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.496+0000 7f848afe5640 20 reqs_thread_entry: start 2024-03-04T18:15:09.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.496+0000 7f85d8119980 20 init_complete bucket index max shards: 1 2024-03-04T18:15:09.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.496+0000 7f85d8119980 20 Filter name: none 2024-03-04T18:15:09.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.496+0000 7f8488fe1640 20 reqs_thread_entry: start 2024-03-04T18:15:09.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.496+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:15:09.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.500+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.500+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.504+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.504+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:15:09.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.508+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.508+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.508+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.512+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:15:09.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.512+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.516+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.516+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.520+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:15:09.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.520+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.520+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.520+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.524+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:15:09.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.524+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.524+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.528+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.528+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:15:09.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.532+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.532+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.532+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.532+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:15:09.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.536+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.540+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.540+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.544+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:15:09.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.544+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.544+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.544+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.548+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:15:09.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.552+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.552+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.552+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.556+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:15:09.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.560+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.560+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.564+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.564+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:15:09.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.568+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.568+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.568+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.568+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:15:09.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.572+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.572+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.572+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.576+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:15:09.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.580+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.580+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.580+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.584+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:15:09.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.588+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.588+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.588+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.588+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:15:09.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.592+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.596+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.596+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.600+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:15:09.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.600+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.600+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.600+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.604+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:15:09.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.608+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.608+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.608+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.608+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:15:09.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.612+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.616+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.616+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.616+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:15:09.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.620+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.620+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.620+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.624+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:15:09.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.628+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.628+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.628+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.632+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:15:09.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.636+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.636+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.640+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.640+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:15:09.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.644+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.644+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.648+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.648+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:15:09.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.652+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.652+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.652+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.656+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:15:09.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.656+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.660+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.660+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.664+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:15:09.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.664+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.664+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.668+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.668+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:15:09.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.672+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.676+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.676+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.680+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:15:09.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.684+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.684+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.684+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.688+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:15:09.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.692+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.692+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.692+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.696+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:15:09.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.700+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.700+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.700+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.704+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:15:09.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.708+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.708+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.708+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.712+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:15:09.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.716+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.716+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.716+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.720+0000 7f85d8119980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:15:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.724+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.724+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:09.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.724+0000 7f85d8119980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:09.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.728+0000 7f848bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:15:09.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.732+0000 7f85d8119980 20 remove_watcher() i=0 2024-03-04T18:15:09.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.732+0000 7f85d8119980 2 removed watcher, disabling cache 2024-03-04T18:15:09.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.736+0000 7f85d8119980 20 remove_watcher() i=1 2024-03-04T18:15:09.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.740+0000 7f85d8119980 20 remove_watcher() i=2 2024-03-04T18:15:09.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.744+0000 7f85d8119980 20 remove_watcher() i=3 2024-03-04T18:15:09.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.748+0000 7f85d8119980 20 remove_watcher() i=4 2024-03-04T18:15:09.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.752+0000 7f85d8119980 20 remove_watcher() i=5 2024-03-04T18:15:09.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.752+0000 7f85d8119980 20 remove_watcher() i=6 2024-03-04T18:15:09.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.756+0000 7f85d8119980 20 remove_watcher() i=7 2024-03-04T18:15:09.780 DEBUG:teuthology.orchestra.run.smithi137:> radosgw-admin gc process --include-all 2024-03-04T18:15:09.852 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:15:09.852 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:15:09.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.892+0000 7faead49f980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.896+0000 7faead49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:09.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.896+0000 7faead49f980 20 realm 2024-03-04T18:15:09.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.896+0000 7faead49f980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.896+0000 7faead49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:09.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.896+0000 7faead49f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:15:09.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.896+0000 7faead49f980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.896+0000 7faead49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:09.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.896+0000 7faead49f980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.896+0000 7faead49f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:15:09.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.896+0000 7faead49f980 20 rados->read ofs=0 len=0 2024-03-04T18:15:09.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.900+0000 7faead49f980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:15:09.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.900+0000 7faead49f980 20 searching for the correct realm 2024-03-04T18:15:09.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:09.904+0000 7faeab7b8640 0 -- 172.21.15.137:0/3793012736 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fae9c0f2c90 msgr2=0x7fae9c108a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:15:10.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.120+0000 7faeab7b8640 0 -- 172.21.15.137:0/3793012736 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x5637aed02f00 msgr2=0x5637aed053c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:15:10.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.128+0000 7faead49f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:15:10.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.128+0000 7faead49f980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:15:10.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.128+0000 7faead49f980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:15:10.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.128+0000 7faead49f980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:15:10.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.128+0000 7faead49f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:15:10.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.128+0000 7faead49f980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:15:10.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.128+0000 7faead49f980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:10.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:15:10.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:15:10.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 zone default found 2024-03-04T18:15:10.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 4 Realm: () 2024-03-04T18:15:10.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:15:10.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:15:10.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:15:10.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 zonegroup default 2024-03-04T18:15:10.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:10.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:10.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:10.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 started sync module instance, tier type = 2024-03-04T18:15:10.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.132+0000 7faead49f980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:15:10.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.156+0000 7faead49f980 20 add_watcher() i=0 2024-03-04T18:15:10.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.156+0000 7faead49f980 20 add_watcher() i=1 2024-03-04T18:15:10.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.156+0000 7faead49f980 20 add_watcher() i=2 2024-03-04T18:15:10.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.156+0000 7faead49f980 20 add_watcher() i=3 2024-03-04T18:15:10.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.156+0000 7faead49f980 20 add_watcher() i=4 2024-03-04T18:15:10.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.156+0000 7faead49f980 20 add_watcher() i=5 2024-03-04T18:15:10.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.160+0000 7faead49f980 20 add_watcher() i=6 2024-03-04T18:15:10.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.164+0000 7faead49f980 20 add_watcher() i=7 2024-03-04T18:15:10.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.164+0000 7faead49f980 2 all 8 watchers are set, enabling cache 2024-03-04T18:15:10.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.168+0000 7fad617e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:15:10.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.168+0000 7faead49f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:15:10.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.168+0000 7faead49f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:15:10.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.252+0000 7facdbfff640 20 reqs_thread_entry: start 2024-03-04T18:15:10.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.252+0000 7faead49f980 20 init_complete bucket index max shards: 1 2024-03-04T18:15:10.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.252+0000 7faead49f980 20 Filter name: none 2024-03-04T18:15:10.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.252+0000 7facd9ffb640 20 reqs_thread_entry: start 2024-03-04T18:15:10.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.252+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:15:10.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.256+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.256+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.260+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.260+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:15:10.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.264+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.264+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.264+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.268+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:15:10.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.272+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.272+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.272+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.272+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:15:10.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.276+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.280+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.280+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.284+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:15:10.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.284+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.288+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.288+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.288+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:15:10.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.292+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.292+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.292+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.296+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:15:10.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.300+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.300+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.300+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.304+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:15:10.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.304+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.308+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.308+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.308+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:15:10.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.312+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.312+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.316+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.316+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:15:10.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.320+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.320+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.320+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.324+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:15:10.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.328+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.328+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.328+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.332+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:15:10.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.332+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.332+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.332+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.336+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:15:10.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.336+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.340+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.340+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.344+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:15:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.344+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.348+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.348+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.348+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:15:10.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.352+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.352+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.356+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.356+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:15:10.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.360+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.360+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.364+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.364+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:15:10.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.368+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.372+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.372+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.372+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:15:10.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.376+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.376+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.376+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.376+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:15:10.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.380+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.384+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.384+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.384+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:15:10.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.388+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.388+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.388+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.392+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:15:10.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.396+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.396+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.396+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.400+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:15:10.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.404+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.404+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.404+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.408+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:15:10.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.412+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.412+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.412+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.416+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:15:10.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.420+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.420+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.420+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.424+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:15:10.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.428+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.428+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.428+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.432+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:15:10.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.436+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.436+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.436+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.440+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:15:10.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.444+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.444+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.444+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.444+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:15:10.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.448+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.452+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.452+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.452+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:15:10.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.456+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.456+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.456+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.460+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:15:10.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.464+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.464+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.464+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.468+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:15:10.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.472+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.472+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.472+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.472+0000 7faead49f980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:15:10.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.476+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.476+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.476+0000 7faead49f980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.480+0000 7fad617e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:15:10.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.484+0000 7faead49f980 20 remove_watcher() i=0 2024-03-04T18:15:10.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.484+0000 7faead49f980 2 removed watcher, disabling cache 2024-03-04T18:15:10.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.488+0000 7faead49f980 20 remove_watcher() i=1 2024-03-04T18:15:10.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.488+0000 7faead49f980 20 remove_watcher() i=2 2024-03-04T18:15:10.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.492+0000 7faead49f980 20 remove_watcher() i=3 2024-03-04T18:15:10.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.496+0000 7faead49f980 20 remove_watcher() i=4 2024-03-04T18:15:10.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.496+0000 7faead49f980 20 remove_watcher() i=5 2024-03-04T18:15:10.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.500+0000 7faead49f980 20 remove_watcher() i=6 2024-03-04T18:15:10.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.504+0000 7faead49f980 20 remove_watcher() i=7 2024-03-04T18:15:10.527 DEBUG:teuthology.orchestra.run.smithi137:> radosgw-admin gc process --include-all 2024-03-04T18:15:10.599 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:15:10.599 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:15:10.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.644+0000 7f412f1c3980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.644+0000 7f412f1c3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.644+0000 7f412f1c3980 20 realm 2024-03-04T18:15:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.644+0000 7f412f1c3980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.644+0000 7f412f1c3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:10.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.644+0000 7f412f1c3980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:15:10.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.644+0000 7f412f1c3980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.644+0000 7f412f1c3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:10.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.644+0000 7f412f1c3980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.644+0000 7f412f1c3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:15:10.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.644+0000 7f412f1c3980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.648+0000 7f412f1c3980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:15:10.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.648+0000 7f412f1c3980 20 searching for the correct realm 2024-03-04T18:15:10.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.652+0000 7f412d4dc640 0 -- 172.21.15.137:0/2522229831 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55fee018a680 msgr2=0x55fee018cb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:15:10.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.660+0000 7f412c4da640 0 -- 172.21.15.137:0/2522229831 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x55fee0185930 msgr2=0x55fee0187d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:15:10.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:15:10.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:15:10.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:15:10.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:15:10.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:15:10.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:15:10.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:10.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:15:10.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:15:10.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 zone default found 2024-03-04T18:15:10.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 4 Realm: () 2024-03-04T18:15:10.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:15:10.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:15:10.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:15:10.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 zonegroup default 2024-03-04T18:15:10.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:10.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:10.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 rados->read ofs=0 len=0 2024-03-04T18:15:10.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:10.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 started sync module instance, tier type = 2024-03-04T18:15:10.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.668+0000 7f412f1c3980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:15:10.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.688+0000 7f412f1c3980 20 add_watcher() i=0 2024-03-04T18:15:10.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.688+0000 7f412f1c3980 20 add_watcher() i=1 2024-03-04T18:15:10.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.688+0000 7f412f1c3980 20 add_watcher() i=2 2024-03-04T18:15:10.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.688+0000 7f412f1c3980 20 add_watcher() i=3 2024-03-04T18:15:10.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.688+0000 7f412f1c3980 20 add_watcher() i=4 2024-03-04T18:15:10.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.688+0000 7f412f1c3980 20 add_watcher() i=5 2024-03-04T18:15:10.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.688+0000 7f412f1c3980 20 add_watcher() i=6 2024-03-04T18:15:10.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.692+0000 7f412f1c3980 20 add_watcher() i=7 2024-03-04T18:15:10.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.692+0000 7f412f1c3980 2 all 8 watchers are set, enabling cache 2024-03-04T18:15:10.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.692+0000 7f3fe2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:15:10.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.696+0000 7f412f1c3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:15:10.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.696+0000 7f412f1c3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:15:10.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.772+0000 7f3fe1fe3640 20 reqs_thread_entry: start 2024-03-04T18:15:10.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.772+0000 7f412f1c3980 20 init_complete bucket index max shards: 1 2024-03-04T18:15:10.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.772+0000 7f412f1c3980 20 Filter name: none 2024-03-04T18:15:10.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.772+0000 7f3f537fe640 20 reqs_thread_entry: start 2024-03-04T18:15:10.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.772+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:15:10.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.776+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.776+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.776+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.780+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:15:10.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.784+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.784+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.784+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.784+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:15:10.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.788+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.788+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.788+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.792+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:15:10.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.792+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.792+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.792+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.796+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:15:10.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.800+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.800+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.804+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.804+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:15:10.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.808+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.808+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.808+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.812+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:15:10.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.816+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.816+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.816+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.820+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:15:10.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.824+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.828+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.828+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.828+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:15:10.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.836+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.836+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.836+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.836+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:15:10.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.840+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.840+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.844+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.844+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:15:10.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.848+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.848+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.852+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.852+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:15:10.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.856+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.860+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.860+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.860+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:15:10.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.864+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.868+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.868+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.872+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:15:10.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.872+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.872+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.872+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.876+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:15:10.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.880+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.880+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.880+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.884+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:15:10.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.884+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.888+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.888+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.892+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:15:10.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.892+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.896+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.896+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.896+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:15:10.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.900+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.900+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.904+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.904+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:15:10.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.908+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.908+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.908+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.912+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:15:10.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.916+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.916+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.916+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.920+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:15:10.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.924+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.924+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.924+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.924+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:15:10.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.928+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.928+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.932+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.932+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:15:10.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.936+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.936+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.936+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.940+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:15:10.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.944+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.944+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.948+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.948+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:15:10.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.952+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.956+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.956+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.960+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:15:10.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.960+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.964+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.964+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.964+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:15:10.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.968+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.968+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.972+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.972+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:15:10.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.976+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.976+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.980+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.980+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:15:10.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.984+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.984+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.988+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.988+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:15:10.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.992+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.992+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:10.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.992+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:10.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:10.996+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:15:11.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.000+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.000+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.000+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.004+0000 7f412f1c3980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:15:11.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.008+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.008+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.008+0000 7f412f1c3980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.012+0000 7f3fe2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:15:11.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.016+0000 7f412f1c3980 20 remove_watcher() i=0 2024-03-04T18:15:11.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.016+0000 7f412f1c3980 2 removed watcher, disabling cache 2024-03-04T18:15:11.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.020+0000 7f412f1c3980 20 remove_watcher() i=1 2024-03-04T18:15:11.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.020+0000 7f412f1c3980 20 remove_watcher() i=2 2024-03-04T18:15:11.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.024+0000 7f412f1c3980 20 remove_watcher() i=3 2024-03-04T18:15:11.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.024+0000 7f412f1c3980 20 remove_watcher() i=4 2024-03-04T18:15:11.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.028+0000 7f412f1c3980 20 remove_watcher() i=5 2024-03-04T18:15:11.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.028+0000 7f412f1c3980 20 remove_watcher() i=6 2024-03-04T18:15:11.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.032+0000 7f412f1c3980 20 remove_watcher() i=7 2024-03-04T18:15:11.053 DEBUG:teuthology.orchestra.run.smithi137:> radosgw-admin gc process --include-all 2024-03-04T18:15:11.123 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:15:11.123 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:15:11.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.164+0000 7f9d7e2b0980 20 rados->read ofs=0 len=0 2024-03-04T18:15:11.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.164+0000 7f9d7e2b0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:11.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.164+0000 7f9d7e2b0980 20 realm 2024-03-04T18:15:11.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.164+0000 7f9d7e2b0980 20 rados->read ofs=0 len=0 2024-03-04T18:15:11.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.168+0000 7f9d7e2b0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:11.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.168+0000 7f9d7e2b0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:15:11.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.168+0000 7f9d7e2b0980 20 rados->read ofs=0 len=0 2024-03-04T18:15:11.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.168+0000 7f9d7e2b0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:11.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.168+0000 7f9d7e2b0980 20 rados->read ofs=0 len=0 2024-03-04T18:15:11.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.168+0000 7f9d7e2b0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:15:11.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.168+0000 7f9d7e2b0980 20 rados->read ofs=0 len=0 2024-03-04T18:15:11.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.168+0000 7f9d7e2b0980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:15:11.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.168+0000 7f9d7e2b0980 20 searching for the correct realm 2024-03-04T18:15:11.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.176+0000 7f9d7c5c9640 0 -- 172.21.15.137:0/3909007644 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f9d6c05ddc0 msgr2=0x7f9d6c06d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:15:11.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.392+0000 7f9d7c5c9640 0 -- 172.21.15.137:0/3909007644 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x557921487870 msgr2=0x557921489d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:15:11.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.400+0000 7f9d7e2b0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:15:11.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.400+0000 7f9d7e2b0980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:15:11.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.400+0000 7f9d7e2b0980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:15:11.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.400+0000 7f9d7e2b0980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:15:11.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.400+0000 7f9d7e2b0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:15:11.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.400+0000 7f9d7e2b0980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:15:11.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.400+0000 7f9d7e2b0980 20 rados->read ofs=0 len=0 2024-03-04T18:15:11.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.400+0000 7f9d7e2b0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:11.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.400+0000 7f9d7e2b0980 20 rados->read ofs=0 len=0 2024-03-04T18:15:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.400+0000 7f9d7e2b0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:15:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.400+0000 7f9d7e2b0980 20 rados->read ofs=0 len=0 2024-03-04T18:15:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:15:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 20 zone default found 2024-03-04T18:15:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 4 Realm: () 2024-03-04T18:15:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:15:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:15:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:15:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 20 zonegroup default 2024-03-04T18:15:11.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 20 rados->read ofs=0 len=0 2024-03-04T18:15:11.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:11.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 20 rados->read ofs=0 len=0 2024-03-04T18:15:11.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:11.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 20 rados->read ofs=0 len=0 2024-03-04T18:15:11.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:15:11.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 20 started sync module instance, tier type = 2024-03-04T18:15:11.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.404+0000 7f9d7e2b0980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:15:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.428+0000 7f9d7e2b0980 20 add_watcher() i=0 2024-03-04T18:15:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.428+0000 7f9d7e2b0980 20 add_watcher() i=1 2024-03-04T18:15:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.428+0000 7f9d7e2b0980 20 add_watcher() i=2 2024-03-04T18:15:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.428+0000 7f9d7e2b0980 20 add_watcher() i=3 2024-03-04T18:15:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.428+0000 7f9d7e2b0980 20 add_watcher() i=4 2024-03-04T18:15:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.428+0000 7f9d7e2b0980 20 add_watcher() i=5 2024-03-04T18:15:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.428+0000 7f9d7e2b0980 20 add_watcher() i=6 2024-03-04T18:15:11.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.428+0000 7f9d7e2b0980 20 add_watcher() i=7 2024-03-04T18:15:11.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.428+0000 7f9d7e2b0980 2 all 8 watchers are set, enabling cache 2024-03-04T18:15:11.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.432+0000 7f9c31fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:15:11.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.436+0000 7f9d7e2b0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:15:11.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.436+0000 7f9d7e2b0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:15:11.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.524+0000 7f9c30fe1640 20 reqs_thread_entry: start 2024-03-04T18:15:11.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.524+0000 7f9d7e2b0980 20 init_complete bucket index max shards: 1 2024-03-04T18:15:11.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.524+0000 7f9d7e2b0980 20 Filter name: none 2024-03-04T18:15:11.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.524+0000 7f9baa7fc640 20 reqs_thread_entry: start 2024-03-04T18:15:11.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.524+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:15:11.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.528+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.528+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.528+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.532+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:15:11.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.536+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.536+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.536+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.540+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:15:11.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.544+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.544+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.544+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.548+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:15:11.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.552+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.552+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.552+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.556+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:15:11.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.556+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.560+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.560+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.560+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:15:11.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.564+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.568+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.568+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.568+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:15:11.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.572+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.572+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.572+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.576+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:15:11.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.580+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.580+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.580+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.584+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:15:11.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.588+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.588+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.592+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.592+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:15:11.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.596+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.600+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.600+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.600+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:15:11.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.604+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.604+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.608+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.608+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:15:11.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.612+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.612+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.612+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.616+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:15:11.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.620+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.620+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.620+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.624+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:15:11.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.628+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.628+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.628+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.632+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:15:11.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.632+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.636+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.636+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.640+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:15:11.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.640+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.644+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.644+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.644+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:15:11.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.648+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.652+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.652+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.656+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:15:11.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.656+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.660+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.660+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.660+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:15:11.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.664+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.664+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.664+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.668+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:15:11.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.668+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.668+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.668+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.672+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:15:11.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.676+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.676+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.680+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.684+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:15:11.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.688+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.688+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.688+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.692+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:15:11.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.696+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.696+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.696+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.700+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:15:11.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.704+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.704+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.704+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.708+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:15:11.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.712+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.712+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.712+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.716+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:15:11.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.720+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.720+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.720+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.720+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:15:11.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.724+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.728+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.728+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.732+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:15:11.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.736+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.736+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.736+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.740+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:15:11.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.744+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.744+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.744+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.748+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:15:11.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.752+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.752+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.752+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.756+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:15:11.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.756+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.760+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.760+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.764+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:15:11.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.768+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.768+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:15:11.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.768+0000 7f9d7e2b0980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:15:11.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.776+0000 7f9c31fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:15:11.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.780+0000 7f9d7e2b0980 20 remove_watcher() i=0 2024-03-04T18:15:11.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.780+0000 7f9d7e2b0980 2 removed watcher, disabling cache 2024-03-04T18:15:11.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.780+0000 7f9d7e2b0980 20 remove_watcher() i=1 2024-03-04T18:15:11.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.784+0000 7f9d7e2b0980 20 remove_watcher() i=2 2024-03-04T18:15:11.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.788+0000 7f9d7e2b0980 20 remove_watcher() i=3 2024-03-04T18:15:11.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.788+0000 7f9d7e2b0980 20 remove_watcher() i=4 2024-03-04T18:15:11.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.792+0000 7f9d7e2b0980 20 remove_watcher() i=5 2024-03-04T18:15:11.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.792+0000 7f9d7e2b0980 20 remove_watcher() i=6 2024-03-04T18:15:11.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:15:11.796+0000 7f9d7e2b0980 20 remove_watcher() i=7 2024-03-04T18:15:11.818 DEBUG:teuthology.orchestra.run.smithi137:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests ObjectTest 2024-03-04T18:15:12.521 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :clean 2024-03-04T18:15:13.218 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :compileJava 2024-03-04T18:15:13.219 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :processResources 2024-03-04T18:15:13.219 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :classes 2024-03-04T18:15:13.819 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:13.819 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :compileTestJava 2024-03-04T18:15:13.820 INFO:teuthology.orchestra.run.smithi137.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-03-04T18:15:13.820 INFO:teuthology.orchestra.run.smithi137.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-03-04T18:15:13.821 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:13.821 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :processTestResources NO-SOURCE 2024-03-04T18:15:13.821 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :testClasses 2024-03-04T18:15:15.819 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:15.827 INFO:teuthology.orchestra.run.smithi137.stdout:> Task :test 2024-03-04T18:15:15.827 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:15.827 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests STANDARD_OUT 2024-03-04T18:15:15.829 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04 18:15:15,055 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-03-04T18:15:15.829 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:15.829 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound STARTED 2024-03-04T18:15:15.829 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:15.829 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound PASSED 2024-03-04T18:15:16.418 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:16.419 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone STARTED 2024-03-04T18:15:16.618 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:16.619 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone PASSED 2024-03-04T18:15:17.119 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:17.119 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty STARTED 2024-03-04T18:15:17.319 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:17.319 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty PASSED 2024-03-04T18:15:17.819 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:17.819 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI STARTED 2024-03-04T18:15:29.919 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:29.919 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI PASSED 2024-03-04T18:15:30.319 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:30.319 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI STARTED 2024-03-04T18:15:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:30.519 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI PASSED 2024-03-04T18:15:31.019 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:31.019 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI STARTED 2024-03-04T18:15:31.419 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:31.419 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI PASSED 2024-03-04T18:15:31.919 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:31.919 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI STARTED 2024-03-04T18:15:31.919 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:31.919 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI PASSED 2024-03-04T18:15:32.119 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:32.119 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI STARTED 2024-03-04T18:15:32.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:32.218 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI PASSED 2024-03-04T18:15:32.618 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:32.619 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB STARTED 2024-03-04T18:15:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:41.719 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB PASSED 2024-03-04T18:15:42.119 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:42.119 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b STARTED 2024-03-04T18:15:42.419 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:42.419 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b PASSED 2024-03-04T18:15:42.919 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:42.919 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb STARTED 2024-03-04T18:15:43.219 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:43.219 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb PASSED 2024-03-04T18:15:43.719 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:43.719 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC STARTED 2024-03-04T18:15:43.919 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:43.919 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC PASSED 2024-03-04T18:15:44.319 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:44.319 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 STARTED 2024-03-04T18:15:44.519 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:44.519 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 PASSED 2024-03-04T18:15:44.918 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:44.919 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey STARTED 2024-03-04T18:15:45.118 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:45.118 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey PASSED 2024-03-04T18:15:45.518 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:45.519 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 STARTED 2024-03-04T18:15:45.718 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:45.719 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 PASSED 2024-03-04T18:15:46.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:46.219 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete STARTED 2024-03-04T18:15:46.519 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:46.519 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete PASSED 2024-03-04T18:15:46.918 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:15:46.918 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA STARTED 2024-03-04T18:16:09.419 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:16:09.419 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA PASSED 2024-03-04T18:16:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:16:10.320 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI STARTED 2024-03-04T18:17:00.519 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:17:00.519 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI PASSED 2024-03-04T18:17:01.418 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:17:01.419 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI STARTED 2024-03-04T18:17:01.518 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:17:01.519 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI PASSED 2024-03-04T18:17:01.918 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:17:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI STARTED 2024-03-04T18:17:02.118 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:17:02.119 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI PASSED 2024-03-04T18:17:02.819 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:17:02.819 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI STARTED 2024-03-04T18:17:16.119 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:17:16.119 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI PASSED 2024-03-04T18:17:16.619 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:17:16.619 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI STARTED 2024-03-04T18:17:16.718 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:17:16.719 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI PASSED 2024-03-04T18:17:16.919 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:17:16.919 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI STARTED 2024-03-04T18:17:33.819 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:17:33.819 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI PASSED 2024-03-04T18:17:36.919 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:17:36.919 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests STANDARD_OUT 2024-03-04T18:17:36.919 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04 18:17:36,852 [s3-transfer-manager-worker-9] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-03-04T18:17:37.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04 18:17:36,946 [s3-transfer-manager-worker-10] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-03-04T18:17:39.119 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:17:39.119 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 STARTED 2024-03-04T18:18:04.519 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:18:04.519 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 PASSED 2024-03-04T18:18:05.318 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:18:05.319 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI STARTED 2024-03-04T18:18:16.919 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:18:16.919 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI PASSED 2024-03-04T18:18:17.318 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:18:17.318 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI STARTED 2024-03-04T18:19:50.419 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:19:50.420 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI PASSED 2024-03-04T18:19:51.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:19:51.019 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI STARTED 2024-03-04T18:20:07.619 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:07.620 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI PASSED 2024-03-04T18:20:08.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:08.018 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI STARTED 2024-03-04T18:20:20.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:20.219 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI PASSED 2024-03-04T18:20:20.718 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:20.718 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause STARTED 2024-03-04T18:20:26.318 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:26.318 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause PASSED 2024-03-04T18:20:40.318 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:40.319 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound STARTED 2024-03-04T18:20:41.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:41.018 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound PASSED 2024-03-04T18:20:41.918 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:41.918 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch STARTED 2024-03-04T18:20:43.519 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:43.519 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch PASSED 2024-03-04T18:20:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:45.219 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort STARTED 2024-03-04T18:20:45.418 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:45.418 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort PASSED 2024-03-04T18:20:45.818 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:45.818 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid STARTED 2024-03-04T18:20:46.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:46.018 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid PASSED 2024-03-04T18:20:46.518 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:46.519 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None STARTED 2024-03-04T18:20:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:46.618 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None PASSED 2024-03-04T18:20:47.118 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:47.118 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable STARTED 2024-03-04T18:20:47.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:47.218 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable PASSED 2024-03-04T18:20:47.718 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:47.718 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes STARTED 2024-03-04T18:20:47.918 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:47.918 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes PASSED 2024-03-04T18:20:48.419 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:48.419 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot STARTED 2024-03-04T18:20:48.918 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:48.919 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot PASSED 2024-03-04T18:20:49.618 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:49.618 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist STARTED 2024-03-04T18:20:50.118 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:50.119 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist PASSED 2024-03-04T18:20:50.818 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:50.818 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage STARTED 2024-03-04T18:20:51.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:51.218 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage PASSED 2024-03-04T18:20:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:51.918 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList STARTED 2024-03-04T18:20:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:52.319 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList PASSED 2024-03-04T18:20:53.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:53.018 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList STARTED 2024-03-04T18:20:53.518 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:53.518 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList PASSED 2024-03-04T18:20:54.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:54.218 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList STARTED 2024-03-04T18:20:54.618 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:54.618 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList PASSED 2024-03-04T18:20:55.318 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:55.318 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable STARTED 2024-03-04T18:20:55.818 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:55.819 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable PASSED 2024-03-04T18:20:56.518 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:56.518 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative STARTED 2024-03-04T18:20:57.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:57.018 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative PASSED 2024-03-04T18:20:57.718 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:57.718 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone STARTED 2024-03-04T18:20:58.118 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:58.118 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone PASSED 2024-03-04T18:20:58.818 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:58.818 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne STARTED 2024-03-04T18:20:59.318 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:20:59.318 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne PASSED 2024-03-04T18:21:00.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:00.018 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero STARTED 2024-03-04T18:21:00.518 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:00.518 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero PASSED 2024-03-04T18:21:01.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:01.218 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt STARTED 2024-03-04T18:21:01.618 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:01.618 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt PASSED 2024-03-04T18:21:02.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:02.218 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic STARTED 2024-03-04T18:21:02.518 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:02.518 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic PASSED 2024-03-04T18:21:03.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:03.218 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt STARTED 2024-03-04T18:21:03.618 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:03.618 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt PASSED 2024-03-04T18:21:04.418 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:04.419 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic STARTED 2024-03-04T18:21:04.818 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:04.818 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic PASSED 2024-03-04T18:21:05.518 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:05.518 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist STARTED 2024-03-04T18:21:06.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:06.018 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist PASSED 2024-03-04T18:21:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:06.718 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist STARTED 2024-03-04T18:21:07.118 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:07.118 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist PASSED 2024-03-04T18:21:08.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:08.018 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist STARTED 2024-03-04T18:21:08.518 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:08.518 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist PASSED 2024-03-04T18:21:09.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:09.218 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty STARTED 2024-03-04T18:21:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:09.618 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty PASSED 2024-03-04T18:21:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:10.218 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone STARTED 2024-03-04T18:21:10.618 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:10.618 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone PASSED 2024-03-04T18:21:11.318 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:11.318 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist STARTED 2024-03-04T18:21:11.618 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:11.618 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist PASSED 2024-03-04T18:21:12.318 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:12.318 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable STARTED 2024-03-04T18:21:12.618 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:12.618 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable PASSED 2024-03-04T18:21:13.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:13.218 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket STARTED 2024-03-04T18:21:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:13.418 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket PASSED 2024-03-04T18:21:13.918 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:13.918 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist STARTED 2024-03-04T18:21:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:14.018 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist PASSED 2024-03-04T18:21:14.418 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:14.418 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl STARTED 2024-03-04T18:21:14.718 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:14.718 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl PASSED 2024-03-04T18:21:15.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:15.218 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag STARTED 2024-03-04T18:21:15.418 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:15.418 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag PASSED 2024-03-04T18:21:16.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:16.018 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete STARTED 2024-03-04T18:21:16.618 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:16.618 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete PASSED 2024-03-04T18:21:17.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:17.018 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket STARTED 2024-03-04T18:21:17.118 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:17.118 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket PASSED 2024-03-04T18:21:17.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:17.218 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode STARTED 2024-03-04T18:21:17.518 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:17.518 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode PASSED 2024-03-04T18:21:18.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:18.018 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode STARTED 2024-03-04T18:21:18.218 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:18.218 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode PASSED 2024-03-04T18:21:18.718 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:18.718 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey STARTED 2024-03-04T18:21:19.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:19.018 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey PASSED 2024-03-04T18:21:19.518 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:19.518 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared STARTED 2024-03-04T18:21:19.618 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:19.618 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared PASSED 2024-03-04T18:21:20.118 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:20.118 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile STARTED 2024-03-04T18:21:33.518 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:33.518 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile PASSED 2024-03-04T18:21:34.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:34.018 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testUploadFileList STARTED 2024-03-04T18:21:47.519 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:47.519 INFO:teuthology.orchestra.run.smithi137.stdout:suite > Object tests > ObjectTest.testUploadFileList PASSED 2024-03-04T18:21:48.087 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:48.087 INFO:teuthology.orchestra.run.smithi137.stdout:---------------------------------------------------------------------- 2024-03-04T18:21:48.087 INFO:teuthology.orchestra.run.smithi137.stdout:| Results: SUCCESS (70 tests, 70 successes, 0 failures, 0 skipped) | 2024-03-04T18:21:48.087 INFO:teuthology.orchestra.run.smithi137.stdout:---------------------------------------------------------------------- 2024-03-04T18:21:48.087 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:48.087 INFO:teuthology.orchestra.run.smithi137.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-03-04T18:21:48.087 INFO:teuthology.orchestra.run.smithi137.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-03-04T18:21:48.087 INFO:teuthology.orchestra.run.smithi137.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-03-04T18:21:48.088 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:48.088 INFO:teuthology.orchestra.run.smithi137.stdout:BUILD SUCCESSFUL in 6m 36s 2024-03-04T18:21:48.088 INFO:teuthology.orchestra.run.smithi137.stdout:5 actionable tasks: 5 executed 2024-03-04T18:21:48.095 DEBUG:teuthology.orchestra.run.smithi137:> radosgw-admin gc process --include-all 2024-03-04T18:21:48.120 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:21:48.120 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:21:48.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.157+0000 7f1fda6cf980 20 rados->read ofs=0 len=0 2024-03-04T18:21:48.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.161+0000 7f1fda6cf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:48.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.161+0000 7f1fda6cf980 20 realm 2024-03-04T18:21:48.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.161+0000 7f1fda6cf980 20 rados->read ofs=0 len=0 2024-03-04T18:21:48.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.161+0000 7f1fda6cf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:48.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.161+0000 7f1fda6cf980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:21:48.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.161+0000 7f1fda6cf980 20 rados->read ofs=0 len=0 2024-03-04T18:21:48.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.161+0000 7f1fda6cf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:48.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.161+0000 7f1fda6cf980 20 rados->read ofs=0 len=0 2024-03-04T18:21:48.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.161+0000 7f1fda6cf980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:48.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.161+0000 7f1fda6cf980 20 rados->read ofs=0 len=0 2024-03-04T18:21:48.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.165+0000 7f1fda6cf980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:21:48.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.165+0000 7f1fda6cf980 20 searching for the correct realm 2024-03-04T18:21:48.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.169+0000 7f1fd89e8640 0 -- 172.21.15.137:0/473234277 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f1fd00183b0 msgr2=0x7f1fd0018830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:21:48.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.385+0000 7f1fd89e8640 0 -- 172.21.15.137:0/473234277 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55fda8fdf450 msgr2=0x55fda8fe1910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:48.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.393+0000 7f1fda6cf980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:21:48.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.393+0000 7f1fda6cf980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:21:48.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.393+0000 7f1fda6cf980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:21:48.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.393+0000 7f1fda6cf980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:21:48.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.393+0000 7f1fda6cf980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:21:48.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.393+0000 7f1fda6cf980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:21:48.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.393+0000 7f1fda6cf980 20 rados->read ofs=0 len=0 2024-03-04T18:21:48.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.393+0000 7f1fda6cf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:48.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.393+0000 7f1fda6cf980 20 rados->read ofs=0 len=0 2024-03-04T18:21:48.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:48.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 20 rados->read ofs=0 len=0 2024-03-04T18:21:48.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:21:48.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 20 zone default found 2024-03-04T18:21:48.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 4 Realm: () 2024-03-04T18:21:48.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:21:48.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:21:48.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:21:48.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 20 zonegroup default 2024-03-04T18:21:48.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 20 rados->read ofs=0 len=0 2024-03-04T18:21:48.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:48.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 20 rados->read ofs=0 len=0 2024-03-04T18:21:48.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:48.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 20 rados->read ofs=0 len=0 2024-03-04T18:21:48.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:48.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 20 started sync module instance, tier type = 2024-03-04T18:21:48.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.397+0000 7f1fda6cf980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:21:48.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.421+0000 7f1fda6cf980 20 add_watcher() i=0 2024-03-04T18:21:48.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.421+0000 7f1fda6cf980 20 add_watcher() i=1 2024-03-04T18:21:48.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.421+0000 7f1fda6cf980 20 add_watcher() i=2 2024-03-04T18:21:48.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.421+0000 7f1fda6cf980 20 add_watcher() i=3 2024-03-04T18:21:48.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.421+0000 7f1fda6cf980 20 add_watcher() i=4 2024-03-04T18:21:48.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.421+0000 7f1fda6cf980 20 add_watcher() i=5 2024-03-04T18:21:48.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.421+0000 7f1fda6cf980 20 add_watcher() i=6 2024-03-04T18:21:48.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.425+0000 7f1fda6cf980 20 add_watcher() i=7 2024-03-04T18:21:48.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.425+0000 7f1fda6cf980 2 all 8 watchers are set, enabling cache 2024-03-04T18:21:48.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.429+0000 7f1e8efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:48.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.429+0000 7f1fda6cf980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:21:48.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.429+0000 7f1fda6cf980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:21:48.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.521+0000 7f1e8dfdb640 20 reqs_thread_entry: start 2024-03-04T18:21:48.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.521+0000 7f1fda6cf980 20 init_complete bucket index max shards: 1 2024-03-04T18:21:48.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.525+0000 7f1fda6cf980 20 Filter name: none 2024-03-04T18:21:48.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.525+0000 7f1dff7fe640 20 reqs_thread_entry: start 2024-03-04T18:21:48.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.525+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:21:48.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.525+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.525+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.529+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.529+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:21:48.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.529+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.533+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.533+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.533+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:21:48.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.537+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.541+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.541+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:21:48.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.541+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5716800602928738705', time=2024-03-04T20:19:29.216259+0000, chain.objs.size()=7 2024-03-04T18:21:48.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.541+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~8WhzqysURntX6_6Exply3oNoZ4Uy05R.1 2024-03-04T18:21:48.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.541+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~8WhzqysURntX6_6Exply3oNoZ4Uy05R.1_1 2024-03-04T18:21:48.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.541+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~8WhzqysURntX6_6Exply3oNoZ4Uy05R.1_2 2024-03-04T18:21:48.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.541+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~8WhzqysURntX6_6Exply3oNoZ4Uy05R.2 2024-03-04T18:21:48.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.541+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~8WhzqysURntX6_6Exply3oNoZ4Uy05R.2_1 2024-03-04T18:21:48.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.541+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~8WhzqysURntX6_6Exply3oNoZ4Uy05R.2_2 2024-03-04T18:21:48.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.541+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~8WhzqysURntX6_6Exply3oNoZ4Uy05R.3 2024-03-04T18:21:48.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.549+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.557+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:21:48.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.557+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.557+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.557+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.561+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:21:48.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.565+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.565+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.565+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:21:48.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.565+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.5532770109212533561', time=2024-03-04T20:21:47.677217+0000, chain.objs.size()=9 2024-03-04T18:21:48.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.565+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.134__multipart_key1/data/file.mpg.2~IVMSZEZJ-3nJs2qFUTVwowkecH9MrHJ.1 2024-03-04T18:21:48.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.565+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.134__shadow_key1/data/file.mpg.2~IVMSZEZJ-3nJs2qFUTVwowkecH9MrHJ.1_1 2024-03-04T18:21:48.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.565+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.134__multipart_key1/data/file.mpg.2~IVMSZEZJ-3nJs2qFUTVwowkecH9MrHJ.2 2024-03-04T18:21:48.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.565+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.134__shadow_key1/data/file.mpg.2~IVMSZEZJ-3nJs2qFUTVwowkecH9MrHJ.2_1 2024-03-04T18:21:48.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.565+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.134__multipart_key1/data/file.mpg.2~IVMSZEZJ-3nJs2qFUTVwowkecH9MrHJ.3 2024-03-04T18:21:48.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.569+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.134__shadow_key1/data/file.mpg.2~IVMSZEZJ-3nJs2qFUTVwowkecH9MrHJ.3_1 2024-03-04T18:21:48.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.569+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.134__multipart_key1/data/file.mpg.2~IVMSZEZJ-3nJs2qFUTVwowkecH9MrHJ.4 2024-03-04T18:21:48.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.569+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.134__shadow_key1/data/file.mpg.2~IVMSZEZJ-3nJs2qFUTVwowkecH9MrHJ.4_1 2024-03-04T18:21:48.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.569+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.134__multipart_key1/data/file.mpg.2~IVMSZEZJ-3nJs2qFUTVwowkecH9MrHJ.5 2024-03-04T18:21:48.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.573+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.581+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:21:48.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.581+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.585+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.585+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.585+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:21:48.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.589+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.589+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.589+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.593+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:21:48.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.597+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.597+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.597+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:21:48.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.597+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.1703297028202767488', time=2024-03-04T20:17:00.697093+0000, chain.objs.size()=5 2024-03-04T18:21:48.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.597+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.83__shadow_.TuCuvDzHBhMwnd_-p-Ed2r8coChMuh0_1 2024-03-04T18:21:48.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.597+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.83__shadow_.TuCuvDzHBhMwnd_-p-Ed2r8coChMuh0_2 2024-03-04T18:21:48.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.597+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.83__shadow_.TuCuvDzHBhMwnd_-p-Ed2r8coChMuh0_3 2024-03-04T18:21:48.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.597+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.83__shadow_.TuCuvDzHBhMwnd_-p-Ed2r8coChMuh0_4 2024-03-04T18:21:48.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.597+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.83__shadow_.TuCuvDzHBhMwnd_-p-Ed2r8coChMuh0_5 2024-03-04T18:21:48.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.605+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.609+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:21:48.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.613+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.613+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.613+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.617+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:21:48.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.617+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.621+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.621+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.621+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:21:48.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.625+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.625+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:21:48.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9557042526825788100', time=2024-03-04T20:18:04.828053+0000, chain.objs.size()=5 2024-03-04T18:21:48.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.90__multipart_file2.mpg.2~ed_H5kWX-ghn4_XoECudMneUZ39k5Tv.1 2024-03-04T18:21:48.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.90__multipart_file2.mpg.2~ed_H5kWX-ghn4_XoECudMneUZ39k5Tv.2 2024-03-04T18:21:48.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.90__multipart_file2.mpg.2~ed_H5kWX-ghn4_XoECudMneUZ39k5Tv.3 2024-03-04T18:21:48.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.90__multipart_file2.mpg.2~ed_H5kWX-ghn4_XoECudMneUZ39k5Tv.4 2024-03-04T18:21:48.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.90__multipart_file2.mpg.2~ed_H5kWX-ghn4_XoECudMneUZ39k5Tv.5 2024-03-04T18:21:48.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.1958007452707722224', time=2024-03-04T20:20:20.385937+0000, chain.objs.size()=9 2024-03-04T18:21:48.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.94__multipart_key1.2~l5z-Q8_BLU_fKpsYxZTGk2_K-SIwY1k.1 2024-03-04T18:21:48.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.94__shadow_key1.2~l5z-Q8_BLU_fKpsYxZTGk2_K-SIwY1k.1_1 2024-03-04T18:21:48.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.94__multipart_key1.2~l5z-Q8_BLU_fKpsYxZTGk2_K-SIwY1k.2 2024-03-04T18:21:48.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.94__shadow_key1.2~l5z-Q8_BLU_fKpsYxZTGk2_K-SIwY1k.2_1 2024-03-04T18:21:48.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.94__multipart_key1.2~l5z-Q8_BLU_fKpsYxZTGk2_K-SIwY1k.3 2024-03-04T18:21:48.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.94__shadow_key1.2~l5z-Q8_BLU_fKpsYxZTGk2_K-SIwY1k.3_1 2024-03-04T18:21:48.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.629+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.94__multipart_key1.2~l5z-Q8_BLU_fKpsYxZTGk2_K-SIwY1k.4 2024-03-04T18:21:48.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.633+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.94__shadow_key1.2~l5z-Q8_BLU_fKpsYxZTGk2_K-SIwY1k.4_1 2024-03-04T18:21:48.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.633+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.94__multipart_key1.2~l5z-Q8_BLU_fKpsYxZTGk2_K-SIwY1k.5 2024-03-04T18:21:48.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.641+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.645+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:21:48.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.649+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.649+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.649+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:21:48.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.649+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7298984994388725616', time=2024-03-04T20:20:07.718477+0000, chain.objs.size()=9 2024-03-04T18:21:48.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.649+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.93__multipart_key1.2~mtucGpIcTws4S42k9-cD_OwbAsxMNtT.1 2024-03-04T18:21:48.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.649+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.93__shadow_key1.2~mtucGpIcTws4S42k9-cD_OwbAsxMNtT.1_1 2024-03-04T18:21:48.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.649+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.93__multipart_key1.2~mtucGpIcTws4S42k9-cD_OwbAsxMNtT.2 2024-03-04T18:21:48.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.649+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.93__shadow_key1.2~mtucGpIcTws4S42k9-cD_OwbAsxMNtT.2_1 2024-03-04T18:21:48.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.649+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.93__multipart_key1.2~mtucGpIcTws4S42k9-cD_OwbAsxMNtT.3 2024-03-04T18:21:48.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.649+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.93__shadow_key1.2~mtucGpIcTws4S42k9-cD_OwbAsxMNtT.3_1 2024-03-04T18:21:48.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.649+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.93__multipart_key1.2~mtucGpIcTws4S42k9-cD_OwbAsxMNtT.4 2024-03-04T18:21:48.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.649+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.93__shadow_key1.2~mtucGpIcTws4S42k9-cD_OwbAsxMNtT.4_1 2024-03-04T18:21:48.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.649+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.93__multipart_key1.2~mtucGpIcTws4S42k9-cD_OwbAsxMNtT.5 2024-03-04T18:21:48.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.657+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.661+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:21:48.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.665+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.665+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.665+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.665+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:21:48.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.669+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.669+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.673+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.673+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:21:48.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.677+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.677+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.677+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.677+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:21:48.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.681+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.685+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.685+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.685+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:21:48.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.689+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.689+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.693+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.693+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:21:48.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-03-04T18:21:48.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7828782989863310160', time=2024-03-04T20:16:33.136494+0000, chain.objs.size()=9 2024-03-04T18:21:48.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~Dzbol57oE6mW_nUDhzOkB0ij2PpypgC.1 2024-03-04T18:21:48.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~Dzbol57oE6mW_nUDhzOkB0ij2PpypgC.1_1 2024-03-04T18:21:48.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~Dzbol57oE6mW_nUDhzOkB0ij2PpypgC.2 2024-03-04T18:21:48.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~Dzbol57oE6mW_nUDhzOkB0ij2PpypgC.2_1 2024-03-04T18:21:48.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~Dzbol57oE6mW_nUDhzOkB0ij2PpypgC.3 2024-03-04T18:21:48.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~Dzbol57oE6mW_nUDhzOkB0ij2PpypgC.3_1 2024-03-04T18:21:48.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~Dzbol57oE6mW_nUDhzOkB0ij2PpypgC.4 2024-03-04T18:21:48.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~Dzbol57oE6mW_nUDhzOkB0ij2PpypgC.4_1 2024-03-04T18:21:48.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~Dzbol57oE6mW_nUDhzOkB0ij2PpypgC.5 2024-03-04T18:21:48.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.4332117642664862410', time=2024-03-04T20:16:56.514913+0000, chain.objs.size()=7 2024-03-04T18:21:48.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~2vlga5Ck4fEamDzj-hwFIyx-xHPKAMX.1 2024-03-04T18:21:48.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~2vlga5Ck4fEamDzj-hwFIyx-xHPKAMX.1_1 2024-03-04T18:21:48.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.697+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~2vlga5Ck4fEamDzj-hwFIyx-xHPKAMX.1_2 2024-03-04T18:21:48.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.713+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~2vlga5Ck4fEamDzj-hwFIyx-xHPKAMX.2 2024-03-04T18:21:48.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.713+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~2vlga5Ck4fEamDzj-hwFIyx-xHPKAMX.2_1 2024-03-04T18:21:48.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.713+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~2vlga5Ck4fEamDzj-hwFIyx-xHPKAMX.2_2 2024-03-04T18:21:48.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.713+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~2vlga5Ck4fEamDzj-hwFIyx-xHPKAMX.3 2024-03-04T18:21:48.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.713+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.599622413622836196', time=2024-03-04T20:17:01.065639+0000, chain.objs.size()=7 2024-03-04T18:21:48.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.713+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~f1nxYsmQOpEzjKVtMb_mSJ41S8TNxL1.1 2024-03-04T18:21:48.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.713+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~f1nxYsmQOpEzjKVtMb_mSJ41S8TNxL1.1_1 2024-03-04T18:21:48.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.713+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~f1nxYsmQOpEzjKVtMb_mSJ41S8TNxL1.1_2 2024-03-04T18:21:48.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.713+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~f1nxYsmQOpEzjKVtMb_mSJ41S8TNxL1.2 2024-03-04T18:21:48.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.713+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~f1nxYsmQOpEzjKVtMb_mSJ41S8TNxL1.2_1 2024-03-04T18:21:48.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.713+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~f1nxYsmQOpEzjKVtMb_mSJ41S8TNxL1.2_2 2024-03-04T18:21:48.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.713+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~f1nxYsmQOpEzjKVtMb_mSJ41S8TNxL1.3 2024-03-04T18:21:48.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.721+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.3312956448117597520', time=2024-03-04T20:17:16.298470+0000, chain.objs.size()=9 2024-03-04T18:21:48.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.721+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.88__multipart_key1.2~4Nl7kofNYRWXJj7PmSNkFaAQAWq2w7s.1 2024-03-04T18:21:48.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.721+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.88__shadow_key1.2~4Nl7kofNYRWXJj7PmSNkFaAQAWq2w7s.1_1 2024-03-04T18:21:48.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.721+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.88__multipart_key1.2~4Nl7kofNYRWXJj7PmSNkFaAQAWq2w7s.2 2024-03-04T18:21:48.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.721+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.88__shadow_key1.2~4Nl7kofNYRWXJj7PmSNkFaAQAWq2w7s.2_1 2024-03-04T18:21:48.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.721+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.88__multipart_key1.2~4Nl7kofNYRWXJj7PmSNkFaAQAWq2w7s.3 2024-03-04T18:21:48.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.721+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.88__shadow_key1.2~4Nl7kofNYRWXJj7PmSNkFaAQAWq2w7s.3_1 2024-03-04T18:21:48.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.721+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.88__multipart_key1.2~4Nl7kofNYRWXJj7PmSNkFaAQAWq2w7s.4 2024-03-04T18:21:48.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.721+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.88__shadow_key1.2~4Nl7kofNYRWXJj7PmSNkFaAQAWq2w7s.4_1 2024-03-04T18:21:48.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.721+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.88__multipart_key1.2~4Nl7kofNYRWXJj7PmSNkFaAQAWq2w7s.5 2024-03-04T18:21:48.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.729+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.733+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:21:48.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.737+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.737+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.741+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:21:48.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.741+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.355316890808343548', time=2024-03-04T20:21:33.646402+0000, chain.objs.size()=9 2024-03-04T18:21:48.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.741+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.133__multipart_key1.2~4fMwYxz-9CmiEy0I6Hs7oUdXnTT03gt.1 2024-03-04T18:21:48.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.741+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.133__shadow_key1.2~4fMwYxz-9CmiEy0I6Hs7oUdXnTT03gt.1_1 2024-03-04T18:21:48.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.741+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.133__multipart_key1.2~4fMwYxz-9CmiEy0I6Hs7oUdXnTT03gt.2 2024-03-04T18:21:48.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.741+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.133__shadow_key1.2~4fMwYxz-9CmiEy0I6Hs7oUdXnTT03gt.2_1 2024-03-04T18:21:48.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.741+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.133__multipart_key1.2~4fMwYxz-9CmiEy0I6Hs7oUdXnTT03gt.3 2024-03-04T18:21:48.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.741+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.133__shadow_key1.2~4fMwYxz-9CmiEy0I6Hs7oUdXnTT03gt.3_1 2024-03-04T18:21:48.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.741+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.133__multipart_key1.2~4fMwYxz-9CmiEy0I6Hs7oUdXnTT03gt.4 2024-03-04T18:21:48.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.741+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.133__shadow_key1.2~4fMwYxz-9CmiEy0I6Hs7oUdXnTT03gt.4_1 2024-03-04T18:21:48.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.741+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.133__multipart_key1.2~4fMwYxz-9CmiEy0I6Hs7oUdXnTT03gt.5 2024-03-04T18:21:48.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.745+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.749+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:21:48.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.753+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.753+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.753+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:21:48.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.753+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8038395065179831589', time=2024-03-04T20:16:52.135402+0000, chain.objs.size()=9 2024-03-04T18:21:48.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.753+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~p7c7CSiWAQxY0BR2rksyCOFakux8ndR.1 2024-03-04T18:21:48.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.753+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~p7c7CSiWAQxY0BR2rksyCOFakux8ndR.1_1 2024-03-04T18:21:48.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.753+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~p7c7CSiWAQxY0BR2rksyCOFakux8ndR.2 2024-03-04T18:21:48.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.753+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~p7c7CSiWAQxY0BR2rksyCOFakux8ndR.2_1 2024-03-04T18:21:48.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.753+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~p7c7CSiWAQxY0BR2rksyCOFakux8ndR.3 2024-03-04T18:21:48.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.753+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~p7c7CSiWAQxY0BR2rksyCOFakux8ndR.3_1 2024-03-04T18:21:48.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.757+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~p7c7CSiWAQxY0BR2rksyCOFakux8ndR.4 2024-03-04T18:21:48.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.757+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~p7c7CSiWAQxY0BR2rksyCOFakux8ndR.4_1 2024-03-04T18:21:48.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.757+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~p7c7CSiWAQxY0BR2rksyCOFakux8ndR.5 2024-03-04T18:21:48.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.761+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.765+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:21:48.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.769+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.769+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.769+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.773+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:21:48.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.777+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.777+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.777+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.781+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:21:48.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.785+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.785+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.785+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:21:48.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.785+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12843836296900440741', time=2024-03-04T20:16:09.549004+0000, chain.objs.size()=9 2024-03-04T18:21:48.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.785+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__multipart_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.1 2024-03-04T18:21:48.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.785+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__shadow_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.1_1 2024-03-04T18:21:48.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.785+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__multipart_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.2 2024-03-04T18:21:48.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.785+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__shadow_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.2_1 2024-03-04T18:21:48.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.785+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__multipart_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.3 2024-03-04T18:21:48.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.785+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__shadow_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.3_1 2024-03-04T18:21:48.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.785+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__multipart_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.4 2024-03-04T18:21:48.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.785+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__shadow_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.4_1 2024-03-04T18:21:48.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.785+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__multipart_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.5 2024-03-04T18:21:48.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.793+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.797+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:21:48.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.797+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.801+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.801+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:21:48.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.801+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9094877195504452176', time=2024-03-04T20:15:41.797223+0000, chain.objs.size()=2 2024-03-04T18:21:48.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.801+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.73__shadow_.uLtZASAcS6_q8Umj5xuGQ8s_gRwrSXl_1 2024-03-04T18:21:48.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.801+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.73__shadow_.uLtZASAcS6_q8Umj5xuGQ8s_gRwrSXl_2 2024-03-04T18:21:48.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.805+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.809+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:21:48.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:21:48.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.11731596840623295811', time=2024-03-04T20:18:04.692831+0000, chain.objs.size()=9 2024-03-04T18:21:48.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.90__multipart_file.mpg.2~sedXwffYPPWi8Yhfi84oKuZ8cHHtdv0.1 2024-03-04T18:21:48.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.90__shadow_file.mpg.2~sedXwffYPPWi8Yhfi84oKuZ8cHHtdv0.1_1 2024-03-04T18:21:48.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.90__multipart_file.mpg.2~sedXwffYPPWi8Yhfi84oKuZ8cHHtdv0.2 2024-03-04T18:21:48.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.90__shadow_file.mpg.2~sedXwffYPPWi8Yhfi84oKuZ8cHHtdv0.2_1 2024-03-04T18:21:48.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.90__multipart_file.mpg.2~sedXwffYPPWi8Yhfi84oKuZ8cHHtdv0.3 2024-03-04T18:21:48.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.90__shadow_file.mpg.2~sedXwffYPPWi8Yhfi84oKuZ8cHHtdv0.3_1 2024-03-04T18:21:48.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.90__multipart_file.mpg.2~sedXwffYPPWi8Yhfi84oKuZ8cHHtdv0.4 2024-03-04T18:21:48.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.90__shadow_file.mpg.2~sedXwffYPPWi8Yhfi84oKuZ8cHHtdv0.4_1 2024-03-04T18:21:48.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.90__multipart_file.mpg.2~sedXwffYPPWi8Yhfi84oKuZ8cHHtdv0.5 2024-03-04T18:21:48.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.15181713764722326604', time=2024-03-04T20:18:43.441575+0000, chain.objs.size()=9 2024-03-04T18:21:48.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~LsG49q9s7HBdeJi5mK1eyXaswZ-41sf.1 2024-03-04T18:21:48.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~LsG49q9s7HBdeJi5mK1eyXaswZ-41sf.1_1 2024-03-04T18:21:48.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.813+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~LsG49q9s7HBdeJi5mK1eyXaswZ-41sf.2 2024-03-04T18:21:48.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.821+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~LsG49q9s7HBdeJi5mK1eyXaswZ-41sf.2_1 2024-03-04T18:21:48.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.821+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~LsG49q9s7HBdeJi5mK1eyXaswZ-41sf.3 2024-03-04T18:21:48.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.821+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~LsG49q9s7HBdeJi5mK1eyXaswZ-41sf.3_1 2024-03-04T18:21:48.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.821+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~LsG49q9s7HBdeJi5mK1eyXaswZ-41sf.4 2024-03-04T18:21:48.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.821+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~LsG49q9s7HBdeJi5mK1eyXaswZ-41sf.4_1 2024-03-04T18:21:48.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.821+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~LsG49q9s7HBdeJi5mK1eyXaswZ-41sf.5 2024-03-04T18:21:48.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.829+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.833+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:21:48.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-03-04T18:21:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.13201877294181083302', time=2024-03-04T20:17:00.411373+0000, chain.objs.size()=7 2024-03-04T18:21:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~haf4FEeKx1qlCPjIkzsvHt01CqMLB7L.1 2024-03-04T18:21:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~haf4FEeKx1qlCPjIkzsvHt01CqMLB7L.1_1 2024-03-04T18:21:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~haf4FEeKx1qlCPjIkzsvHt01CqMLB7L.1_2 2024-03-04T18:21:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~haf4FEeKx1qlCPjIkzsvHt01CqMLB7L.2 2024-03-04T18:21:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~haf4FEeKx1qlCPjIkzsvHt01CqMLB7L.2_1 2024-03-04T18:21:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~haf4FEeKx1qlCPjIkzsvHt01CqMLB7L.2_2 2024-03-04T18:21:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~haf4FEeKx1qlCPjIkzsvHt01CqMLB7L.3 2024-03-04T18:21:48.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.8745630650669474819', time=2024-03-04T20:19:17.093959+0000, chain.objs.size()=9 2024-03-04T18:21:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~ECRkov1LytnmzTAFCkFCnGvGpPtkuMy.1 2024-03-04T18:21:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~ECRkov1LytnmzTAFCkFCnGvGpPtkuMy.1_1 2024-03-04T18:21:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~ECRkov1LytnmzTAFCkFCnGvGpPtkuMy.2 2024-03-04T18:21:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~ECRkov1LytnmzTAFCkFCnGvGpPtkuMy.2_1 2024-03-04T18:21:48.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.837+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~ECRkov1LytnmzTAFCkFCnGvGpPtkuMy.3 2024-03-04T18:21:48.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.849+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~ECRkov1LytnmzTAFCkFCnGvGpPtkuMy.3_1 2024-03-04T18:21:48.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.849+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~ECRkov1LytnmzTAFCkFCnGvGpPtkuMy.4 2024-03-04T18:21:48.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.849+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~ECRkov1LytnmzTAFCkFCnGvGpPtkuMy.4_1 2024-03-04T18:21:48.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.849+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~ECRkov1LytnmzTAFCkFCnGvGpPtkuMy.5 2024-03-04T18:21:48.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.853+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.857+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:21:48.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.861+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.861+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.861+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:21:48.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.861+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='2~Z2FGOaH_wuxi9yFuhw9EZgelP9jqtLl', time=2024-03-04T20:15:29.818484+0000, chain.objs.size()=9 2024-03-04T18:21:48.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.861+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.69__multipart_key1.2~Z2FGOaH_wuxi9yFuhw9EZgelP9jqtLl.1 2024-03-04T18:21:48.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.861+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.69__shadow_key1.2~Z2FGOaH_wuxi9yFuhw9EZgelP9jqtLl.1_1 2024-03-04T18:21:48.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.861+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.69__multipart_key1.2~Z2FGOaH_wuxi9yFuhw9EZgelP9jqtLl.2 2024-03-04T18:21:48.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.861+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.69__shadow_key1.2~Z2FGOaH_wuxi9yFuhw9EZgelP9jqtLl.2_1 2024-03-04T18:21:48.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.861+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.69__multipart_key1.2~Z2FGOaH_wuxi9yFuhw9EZgelP9jqtLl.3 2024-03-04T18:21:48.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.861+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.69__shadow_key1.2~Z2FGOaH_wuxi9yFuhw9EZgelP9jqtLl.3_1 2024-03-04T18:21:48.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.861+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.69__multipart_key1.2~Z2FGOaH_wuxi9yFuhw9EZgelP9jqtLl.4 2024-03-04T18:21:48.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.861+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.69__shadow_key1.2~Z2FGOaH_wuxi9yFuhw9EZgelP9jqtLl.4_1 2024-03-04T18:21:48.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.861+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.69__multipart_key1.2~Z2FGOaH_wuxi9yFuhw9EZgelP9jqtLl.5 2024-03-04T18:21:48.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.873+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.877+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:21:48.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.877+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.877+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.881+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.881+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:21:48.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.885+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.885+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.885+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:21:48.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.885+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__shadow_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.1_1_ca68782f9fb65f5c620c9049f5a434ee', time=2024-03-04T20:16:09.912834+0000, chain.objs.size()=9 2024-03-04T18:21:48.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.885+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__multipart_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.1 2024-03-04T18:21:48.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.885+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__shadow_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.1_1 2024-03-04T18:21:48.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.885+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__multipart_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.2 2024-03-04T18:21:48.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.885+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__shadow_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.2_1 2024-03-04T18:21:48.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.885+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__multipart_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.3 2024-03-04T18:21:48.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.885+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__shadow_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.3_1 2024-03-04T18:21:48.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.889+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__multipart_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.4 2024-03-04T18:21:48.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.889+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__shadow_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.4_1 2024-03-04T18:21:48.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.889+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.81__multipart_key1.2~lFb3-GYysQot9HZ2W8lWj5IxXrPD-cF.5 2024-03-04T18:21:48.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.893+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.897+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:21:48.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-03-04T18:21:48.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12774773030222799015', time=2024-03-04T20:16:47.647497+0000, chain.objs.size()=9 2024-03-04T18:21:48.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~wfVEuFXv-Y0Owwx9K7JdSCgJZika4XH.1 2024-03-04T18:21:48.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~wfVEuFXv-Y0Owwx9K7JdSCgJZika4XH.1_1 2024-03-04T18:21:48.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~wfVEuFXv-Y0Owwx9K7JdSCgJZika4XH.2 2024-03-04T18:21:48.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~wfVEuFXv-Y0Owwx9K7JdSCgJZika4XH.2_1 2024-03-04T18:21:48.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~wfVEuFXv-Y0Owwx9K7JdSCgJZika4XH.3 2024-03-04T18:21:48.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~wfVEuFXv-Y0Owwx9K7JdSCgJZika4XH.3_1 2024-03-04T18:21:48.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~wfVEuFXv-Y0Owwx9K7JdSCgJZika4XH.4 2024-03-04T18:21:48.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__shadow_key1.2~wfVEuFXv-Y0Owwx9K7JdSCgJZika4XH.4_1 2024-03-04T18:21:48.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.84__multipart_key1.2~wfVEuFXv-Y0Owwx9K7JdSCgJZika4XH.5 2024-03-04T18:21:48.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1', time=2024-03-04T20:20:34.759720+0000, chain.objs.size()=1 2024-03-04T18:21:48.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.14 2024-03-04T18:21:48.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1', time=2024-03-04T20:20:35.234735+0000, chain.objs.size()=1 2024-03-04T18:21:48.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.15 2024-03-04T18:21:48.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1', time=2024-03-04T20:20:35.742273+0000, chain.objs.size()=1 2024-03-04T18:21:48.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.901+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.16 2024-03-04T18:21:48.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.909+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1', time=2024-03-04T20:20:35.827299+0000, chain.objs.size()=1 2024-03-04T18:21:48.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.909+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.17 2024-03-04T18:21:48.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.909+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1', time=2024-03-04T20:20:35.970080+0000, chain.objs.size()=1 2024-03-04T18:21:48.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.909+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.18 2024-03-04T18:21:48.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.909+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1', time=2024-03-04T20:20:36.263480+0000, chain.objs.size()=1 2024-03-04T18:21:48.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.909+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.19 2024-03-04T18:21:48.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.909+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1', time=2024-03-04T20:20:36.493156+0000, chain.objs.size()=1 2024-03-04T18:21:48.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.909+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.20 2024-03-04T18:21:48.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.909+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1', time=2024-03-04T20:20:36.582320+0000, chain.objs.size()=26 2024-03-04T18:21:48.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.909+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.1 2024-03-04T18:21:48.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.909+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.2 2024-03-04T18:21:48.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.909+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.3 2024-03-04T18:21:48.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.909+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.4 2024-03-04T18:21:48.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.913+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.5 2024-03-04T18:21:48.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.913+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.6 2024-03-04T18:21:48.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.913+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.7 2024-03-04T18:21:48.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.913+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.8 2024-03-04T18:21:48.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.913+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.9 2024-03-04T18:21:48.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.913+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.10 2024-03-04T18:21:48.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.913+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.11 2024-03-04T18:21:48.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.917+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.12 2024-03-04T18:21:48.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.917+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.13 2024-03-04T18:21:48.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.917+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.4c-Chr6gld7pmqm9YrDaA1VjZT5QXXkO.14 2024-03-04T18:21:48.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.917+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.hEtpEv-DLsXsayetu1rKaiF3zAtw2PVB.15 2024-03-04T18:21:48.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.917+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.ksmizrYurRw3rWQ9wfVCDy0YLcVBT03y.16 2024-03-04T18:21:48.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.917+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.4L3PsDlDxYRBn__EPjoPjmTJVb--GcyY.17 2024-03-04T18:21:48.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.917+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.-hV41rLCoQhz2_5dp7HuOeY0KcZrxNZ2.18 2024-03-04T18:21:48.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.921+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.NEJq5I-wzlVGuFzsov_SuJDBRRoLp4p0.19 2024-03-04T18:21:48.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.921+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.816prl54P5Xq0ybKtZqzGcFshihV6tju.20 2024-03-04T18:21:48.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.921+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.21 2024-03-04T18:21:48.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.921+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.22 2024-03-04T18:21:48.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.921+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.23 2024-03-04T18:21:48.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.921+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.24 2024-03-04T18:21:48.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.921+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.25 2024-03-04T18:21:48.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.925+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.95__multipart_key1.2~KfD0Od2g6an3pQfwkXTEH141zH2oAB1.26 2024-03-04T18:21:48.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.925+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.933+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:21:48.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.933+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-03-04T18:21:48.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12370535323171964543', time=2024-03-04T20:17:36.982527+0000, chain.objs.size()=9 2024-03-04T18:21:48.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.89__multipart_key1.2~ohx35Bft31wneYoeDK9Zy_1leBlucwH.1 2024-03-04T18:21:48.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.89__shadow_key1.2~ohx35Bft31wneYoeDK9Zy_1leBlucwH.1_1 2024-03-04T18:21:48.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.89__multipart_key1.2~ohx35Bft31wneYoeDK9Zy_1leBlucwH.2 2024-03-04T18:21:48.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.89__shadow_key1.2~ohx35Bft31wneYoeDK9Zy_1leBlucwH.2_1 2024-03-04T18:21:48.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.89__multipart_key1.2~ohx35Bft31wneYoeDK9Zy_1leBlucwH.3 2024-03-04T18:21:48.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.89__shadow_key1.2~ohx35Bft31wneYoeDK9Zy_1leBlucwH.3_1 2024-03-04T18:21:48.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.89__multipart_key1.2~ohx35Bft31wneYoeDK9Zy_1leBlucwH.4 2024-03-04T18:21:48.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.89__shadow_key1.2~ohx35Bft31wneYoeDK9Zy_1leBlucwH.4_1 2024-03-04T18:21:48.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.89__multipart_key1.2~ohx35Bft31wneYoeDK9Zy_1leBlucwH.5 2024-03-04T18:21:48.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.12101598878552757780', time=2024-03-04T20:18:55.893648+0000, chain.objs.size()=9 2024-03-04T18:21:48.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~mHpZb3ARRVW6uOVw4VRLOb-apYqRucj.1 2024-03-04T18:21:48.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~mHpZb3ARRVW6uOVw4VRLOb-apYqRucj.1_1 2024-03-04T18:21:48.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.937+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~mHpZb3ARRVW6uOVw4VRLOb-apYqRucj.2 2024-03-04T18:21:48.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.945+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~mHpZb3ARRVW6uOVw4VRLOb-apYqRucj.2_1 2024-03-04T18:21:48.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.945+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~mHpZb3ARRVW6uOVw4VRLOb-apYqRucj.3 2024-03-04T18:21:48.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.945+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~mHpZb3ARRVW6uOVw4VRLOb-apYqRucj.3_1 2024-03-04T18:21:48.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.945+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~mHpZb3ARRVW6uOVw4VRLOb-apYqRucj.4 2024-03-04T18:21:48.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.945+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~mHpZb3ARRVW6uOVw4VRLOb-apYqRucj.4_1 2024-03-04T18:21:48.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.945+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~mHpZb3ARRVW6uOVw4VRLOb-apYqRucj.5 2024-03-04T18:21:48.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.945+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.7366841112406643858', time=2024-03-04T20:19:50.336544+0000, chain.objs.size()=7 2024-03-04T18:21:48.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.945+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~TRj-FsGdvNsLr4aE9Ffeicy0Ay8guem.1 2024-03-04T18:21:48.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.945+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~TRj-FsGdvNsLr4aE9Ffeicy0Ay8guem.1_1 2024-03-04T18:21:48.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.945+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~TRj-FsGdvNsLr4aE9Ffeicy0Ay8guem.1_2 2024-03-04T18:21:48.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.949+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~TRj-FsGdvNsLr4aE9Ffeicy0Ay8guem.2 2024-03-04T18:21:48.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.949+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~TRj-FsGdvNsLr4aE9Ffeicy0Ay8guem.2_1 2024-03-04T18:21:48.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.953+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~TRj-FsGdvNsLr4aE9Ffeicy0Ay8guem.2_2 2024-03-04T18:21:48.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.953+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~TRj-FsGdvNsLr4aE9Ffeicy0Ay8guem.3 2024-03-04T18:21:48.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.961+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.965+0000 7f1fda6cf980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:21:48.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.969+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:48.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.969+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:48.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.969+0000 7f1fda6cf980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-04T18:21:48.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.969+0000 7f1fda6cf980 20 garbage collection: RGWGC::process iterating over entry tag='f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.9274661159547129894', time=2024-03-04T20:19:50.612892+0000, chain.objs.size()=7 2024-03-04T18:21:48.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.969+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~cLQ1kfHCSiizjHFUujTkHDSNfFiuPHc.1 2024-03-04T18:21:48.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.969+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~cLQ1kfHCSiizjHFUujTkHDSNfFiuPHc.1_1 2024-03-04T18:21:48.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.969+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~cLQ1kfHCSiizjHFUujTkHDSNfFiuPHc.1_2 2024-03-04T18:21:48.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.969+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~cLQ1kfHCSiizjHFUujTkHDSNfFiuPHc.2 2024-03-04T18:21:48.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.969+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~cLQ1kfHCSiizjHFUujTkHDSNfFiuPHc.2_1 2024-03-04T18:21:48.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.969+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__shadow_key1.2~cLQ1kfHCSiizjHFUujTkHDSNfFiuPHc.2_2 2024-03-04T18:21:48.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.969+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:f85b0d2e-c665-46d3-ae77-23c29725ed4a.4510.92__multipart_key1.2~cLQ1kfHCSiizjHFUujTkHDSNfFiuPHc.3 2024-03-04T18:21:48.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.977+0000 7f1fda6cf980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-04T18:21:48.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.981+0000 7f1e8efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:48.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.985+0000 7f1fda6cf980 20 remove_watcher() i=0 2024-03-04T18:21:48.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.985+0000 7f1fda6cf980 2 removed watcher, disabling cache 2024-03-04T18:21:48.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.989+0000 7f1fda6cf980 20 remove_watcher() i=1 2024-03-04T18:21:48.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.989+0000 7f1fda6cf980 20 remove_watcher() i=2 2024-03-04T18:21:48.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.993+0000 7f1fda6cf980 20 remove_watcher() i=3 2024-03-04T18:21:48.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.993+0000 7f1fda6cf980 20 remove_watcher() i=4 2024-03-04T18:21:48.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.997+0000 7f1fda6cf980 20 remove_watcher() i=5 2024-03-04T18:21:49.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:48.997+0000 7f1fda6cf980 20 remove_watcher() i=6 2024-03-04T18:21:49.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.001+0000 7f1fda6cf980 20 remove_watcher() i=7 2024-03-04T18:21:49.020 DEBUG:teuthology.orchestra.run.smithi137:> radosgw-admin gc process --include-all 2024-03-04T18:21:49.091 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:21:49.091 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:21:49.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.137+0000 7f45a0b35980 20 rados->read ofs=0 len=0 2024-03-04T18:21:49.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.137+0000 7f45a0b35980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:49.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.137+0000 7f45a0b35980 20 realm 2024-03-04T18:21:49.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.137+0000 7f45a0b35980 20 rados->read ofs=0 len=0 2024-03-04T18:21:49.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.137+0000 7f45a0b35980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:49.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.137+0000 7f45a0b35980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:21:49.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.137+0000 7f45a0b35980 20 rados->read ofs=0 len=0 2024-03-04T18:21:49.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.137+0000 7f45a0b35980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:49.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.137+0000 7f45a0b35980 20 rados->read ofs=0 len=0 2024-03-04T18:21:49.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.141+0000 7f45a0b35980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:49.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.141+0000 7f45a0b35980 20 rados->read ofs=0 len=0 2024-03-04T18:21:49.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.141+0000 7f45a0b35980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:21:49.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.141+0000 7f45a0b35980 20 searching for the correct realm 2024-03-04T18:21:49.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.145+0000 7f459ee4e640 0 -- 172.21.15.137:0/3703291948 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f4590007940 msgr2=0x7f45900f3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:49.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.361+0000 7f459ee4e640 0 -- 172.21.15.137:0/3703291948 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55a0c496fed0 msgr2=0x55a0c4972390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:49.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.369+0000 7f45a0b35980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:21:49.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.369+0000 7f45a0b35980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:21:49.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.369+0000 7f45a0b35980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:21:49.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.369+0000 7f45a0b35980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:21:49.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.369+0000 7f45a0b35980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:21:49.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.369+0000 7f45a0b35980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:21:49.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.369+0000 7f45a0b35980 20 rados->read ofs=0 len=0 2024-03-04T18:21:49.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:49.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 rados->read ofs=0 len=0 2024-03-04T18:21:49.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:49.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 rados->read ofs=0 len=0 2024-03-04T18:21:49.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:21:49.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 zone default found 2024-03-04T18:21:49.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 4 Realm: () 2024-03-04T18:21:49.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:21:49.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:21:49.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:21:49.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 zonegroup default 2024-03-04T18:21:49.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 rados->read ofs=0 len=0 2024-03-04T18:21:49.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:49.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 rados->read ofs=0 len=0 2024-03-04T18:21:49.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:49.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 rados->read ofs=0 len=0 2024-03-04T18:21:49.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:49.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 started sync module instance, tier type = 2024-03-04T18:21:49.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.373+0000 7f45a0b35980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:21:49.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.397+0000 7f45a0b35980 20 add_watcher() i=0 2024-03-04T18:21:49.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.397+0000 7f45a0b35980 20 add_watcher() i=1 2024-03-04T18:21:49.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.397+0000 7f45a0b35980 20 add_watcher() i=2 2024-03-04T18:21:49.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.397+0000 7f45a0b35980 20 add_watcher() i=3 2024-03-04T18:21:49.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.397+0000 7f45a0b35980 20 add_watcher() i=4 2024-03-04T18:21:49.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.397+0000 7f45a0b35980 20 add_watcher() i=5 2024-03-04T18:21:49.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.397+0000 7f45a0b35980 20 add_watcher() i=6 2024-03-04T18:21:49.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.397+0000 7f45a0b35980 20 add_watcher() i=7 2024-03-04T18:21:49.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.397+0000 7f45a0b35980 2 all 8 watchers are set, enabling cache 2024-03-04T18:21:49.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.401+0000 7f4454fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:49.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.405+0000 7f45a0b35980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:21:49.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.405+0000 7f45a0b35980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:21:49.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.493+0000 7f43cf7fe640 20 reqs_thread_entry: start 2024-03-04T18:21:49.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.493+0000 7f45a0b35980 20 init_complete bucket index max shards: 1 2024-03-04T18:21:49.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.493+0000 7f45a0b35980 20 Filter name: none 2024-03-04T18:21:49.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.493+0000 7f43cd7fa640 20 reqs_thread_entry: start 2024-03-04T18:21:49.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.497+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-04T18:21:49.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.497+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.497+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.497+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.501+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-04T18:21:49.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.505+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.505+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.505+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.509+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-04T18:21:49.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.509+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.513+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.513+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.513+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-04T18:21:49.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.517+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.517+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.517+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.521+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-04T18:21:49.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.525+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.525+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.525+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.529+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-04T18:21:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.529+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.529+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.529+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.533+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-04T18:21:49.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.537+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.537+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.537+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.541+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-04T18:21:49.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.541+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.545+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.545+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.545+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-04T18:21:49.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.549+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.553+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.553+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.557+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-04T18:21:49.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.561+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.561+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.561+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.565+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-04T18:21:49.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.569+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.569+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.573+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.573+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-04T18:21:49.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.577+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.577+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.581+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.581+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-04T18:21:49.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.585+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.589+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.589+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.589+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-04T18:21:49.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.593+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.597+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.597+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.601+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-04T18:21:49.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.601+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.605+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.605+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.609+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-04T18:21:49.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.613+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.613+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.613+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.617+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-04T18:21:49.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.621+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.621+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.621+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.625+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-04T18:21:49.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.629+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.629+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.633+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.633+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-04T18:21:49.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.637+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.637+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.641+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.641+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-04T18:21:49.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.645+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.649+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.649+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.653+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-04T18:21:49.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.657+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.657+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.657+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.661+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-04T18:21:49.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.665+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.665+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.665+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.669+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-04T18:21:49.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.673+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.673+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.673+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.677+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-04T18:21:49.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.681+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.681+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.681+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.685+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-04T18:21:49.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.689+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.689+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.689+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.693+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-04T18:21:49.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.697+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.701+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.701+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.705+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-04T18:21:49.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.709+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.709+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.709+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.713+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-04T18:21:49.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.717+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.717+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.717+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.721+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-04T18:21:49.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.725+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.725+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.725+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.729+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-04T18:21:49.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.733+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.733+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.737+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.737+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-04T18:21:49.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.741+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.745+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.745+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.749+0000 7f45a0b35980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-04T18:21:49.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.753+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.753+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-04T18:21:49.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.753+0000 7f45a0b35980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-04T18:21:49.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.761+0000 7f4454fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:49.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.765+0000 7f45a0b35980 20 remove_watcher() i=0 2024-03-04T18:21:49.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.765+0000 7f45a0b35980 2 removed watcher, disabling cache 2024-03-04T18:21:49.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.765+0000 7f45a0b35980 20 remove_watcher() i=1 2024-03-04T18:21:49.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.769+0000 7f45a0b35980 20 remove_watcher() i=2 2024-03-04T18:21:49.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.769+0000 7f45a0b35980 20 remove_watcher() i=3 2024-03-04T18:21:49.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.773+0000 7f45a0b35980 20 remove_watcher() i=4 2024-03-04T18:21:49.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.777+0000 7f45a0b35980 20 remove_watcher() i=5 2024-03-04T18:21:49.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.777+0000 7f45a0b35980 20 remove_watcher() i=6 2024-03-04T18:21:49.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:49.781+0000 7f45a0b35980 20 remove_watcher() i=7 2024-03-04T18:21:49.800 INFO:teuthology.run_tasks:Running task s3tests... 2024-03-04T18:21:49.816 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master', 'storage classes': 'LUKEWARM, FROZEN'}} 2024-03-04T18:21:49.816 INFO:tasks.s3tests:Downloading s3-tests... 2024-03-04T18:21:49.816 INFO:tasks.s3tests:Using branch 'ceph-master' for s3tests 2024-03-04T18:21:49.816 DEBUG:teuthology.orchestra.run.smithi137:> git clone -b ceph-master https://git.ceph.com/s3-tests.git /home/ubuntu/cephtest/s3-tests-client.0 2024-03-04T18:21:49.846 INFO:teuthology.orchestra.run.smithi137.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-client.0'... 2024-03-04T18:21:50.729 INFO:tasks.s3tests:Creating rgw users... 2024-03-04T18:21:50.729 DEBUG:tasks.s3tests:Creating user foo.client.0 on client.0 2024-03-04T18:21:50.730 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name 'Mr. foo.client.0' --email foo.client.0+test@test.test --caps 'user-policy=*' --access-key ZFEULGCABYJCYFVXLEKB --secret eMKpNUy0MXcPyR7gOZ9H/4fIp/P5Ao/1KjEBKBTbuH//3d+Uet80kw== --cluster ceph 2024-03-04T18:21:50.818 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:21:50.819 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:21:50.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:50.857+0000 7f653008d980 20 rados->read ofs=0 len=0 2024-03-04T18:21:50.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:50.861+0000 7f653008d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:50.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:50.861+0000 7f653008d980 20 realm 2024-03-04T18:21:50.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:50.861+0000 7f653008d980 20 rados->read ofs=0 len=0 2024-03-04T18:21:50.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:50.861+0000 7f653008d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:50.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:50.861+0000 7f653008d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:21:50.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:50.861+0000 7f653008d980 20 rados->read ofs=0 len=0 2024-03-04T18:21:50.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:50.861+0000 7f653008d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:50.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:50.861+0000 7f653008d980 20 rados->read ofs=0 len=0 2024-03-04T18:21:50.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:50.861+0000 7f653008d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:50.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:50.861+0000 7f653008d980 20 rados->read ofs=0 len=0 2024-03-04T18:21:50.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:50.861+0000 7f653008d980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:21:50.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:50.861+0000 7f653008d980 20 searching for the correct realm 2024-03-04T18:21:50.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:50.869+0000 7f652e3a6640 0 -- 172.21.15.137:0/494829745 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f6528013030 msgr2=0x7f65280711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:21:51.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.085+0000 7f652e3a6640 0 -- 172.21.15.137:0/494829745 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x557cd84d18a0 msgr2=0x557cd84d3d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:51.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.093+0000 7f653008d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:21:51.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.093+0000 7f653008d980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:21:51.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.093+0000 7f653008d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:21:51.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.093+0000 7f653008d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:21:51.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.093+0000 7f653008d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:21:51.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.093+0000 7f653008d980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:21:51.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.093+0000 7f653008d980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:51.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:51.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:21:51.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 zone default found 2024-03-04T18:21:51.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 4 Realm: () 2024-03-04T18:21:51.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:21:51.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:21:51.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:21:51.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 zonegroup default 2024-03-04T18:21:51.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:51.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:51.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:51.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 started sync module instance, tier type = 2024-03-04T18:21:51.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.097+0000 7f653008d980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:21:51.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.121+0000 7f653008d980 20 add_watcher() i=0 2024-03-04T18:21:51.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.121+0000 7f653008d980 20 add_watcher() i=1 2024-03-04T18:21:51.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.121+0000 7f653008d980 20 add_watcher() i=2 2024-03-04T18:21:51.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.121+0000 7f653008d980 20 add_watcher() i=3 2024-03-04T18:21:51.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.121+0000 7f653008d980 20 add_watcher() i=4 2024-03-04T18:21:51.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.121+0000 7f653008d980 20 add_watcher() i=5 2024-03-04T18:21:51.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.121+0000 7f653008d980 20 add_watcher() i=6 2024-03-04T18:21:51.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.125+0000 7f653008d980 20 add_watcher() i=7 2024-03-04T18:21:51.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.125+0000 7f653008d980 2 all 8 watchers are set, enabling cache 2024-03-04T18:21:51.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.129+0000 7f63df7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:51.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.129+0000 7f653008d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:21:51.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.129+0000 7f653008d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:21:51.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.129+0000 7f653008d980 5 note: GC not initialized 2024-03-04T18:21:51.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.129+0000 7f63de7fc640 20 reqs_thread_entry: start 2024-03-04T18:21:51.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.129+0000 7f653008d980 20 init_complete bucket index max shards: 1 2024-03-04T18:21:51.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.129+0000 7f653008d980 20 Filter name: none 2024-03-04T18:21:51.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.129+0000 7f634ffff640 20 reqs_thread_entry: start 2024-03-04T18:21:51.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.129+0000 7f653008d980 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2024-03-04T18:21:51.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.129+0000 7f653008d980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:51.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2024-03-04T18:21:51.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-03-04T18:21:51.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2024-03-04T18:21:51.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:51.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2024-03-04T18:21:51.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2024-03-04T18:21:51.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 10 cache get: name=default.rgw.meta+users.keys+ZFEULGCABYJCYFVXLEKB : miss 2024-03-04T18:21:51.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:51.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 10 cache put: name=default.rgw.meta+users.keys+ZFEULGCABYJCYFVXLEKB info.flags=0x0 2024-03-04T18:21:51.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 10 adding default.rgw.meta+users.keys+ZFEULGCABYJCYFVXLEKB to cache LRU end 2024-03-04T18:21:51.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 10 cache get: name=default.rgw.meta+users.keys+ZFEULGCABYJCYFVXLEKB : hit (negative entry) 2024-03-04T18:21:51.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.133+0000 7f653008d980 10 cache get: name=default.rgw.meta+users.keys+ZFEULGCABYJCYFVXLEKB : hit (negative entry) 2024-03-04T18:21:51.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.137+0000 7f653008d980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-03-04T18:21:51.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.137+0000 7f653008d980 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-03-04T18:21:51.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.137+0000 7f653008d980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2024-03-04T18:21:51.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.137+0000 7f64117fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627915 cookie 93994693240608 notifier 7578 bl.length()=641 2024-03-04T18:21:51.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.137+0000 7f64117fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-03-04T18:21:51.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.137+0000 7f64117fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-03-04T18:21:51.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.141+0000 7f653008d980 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2024-03-04T18:21:51.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.141+0000 7f653008d980 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2024-03-04T18:21:51.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.141+0000 7f653008d980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.email:foo.client.0+test@test.test, ofs0, ns] 2024-03-04T18:21:51.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.141+0000 7f6410ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627991 cookie 93994693259168 notifier 7578 bl.length()=194 2024-03-04T18:21:51.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.141+0000 7f6410ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2024-03-04T18:21:51.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.141+0000 7f6410ff9640 10 rgw watcher librados: moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2024-03-04T18:21:51.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.149+0000 7f653008d980 10 cache put: name=default.rgw.meta+users.keys+ZFEULGCABYJCYFVXLEKB info.flags=0x7 2024-03-04T18:21:51.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.149+0000 7f653008d980 10 moving default.rgw.meta+users.keys+ZFEULGCABYJCYFVXLEKB to cache LRU end 2024-03-04T18:21:51.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.149+0000 7f653008d980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.keys:ZFEULGCABYJCYFVXLEKB, ofs0, ns] 2024-03-04T18:21:51.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.149+0000 7f6410ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1099511627989 cookie 93994693250256 notifier 7578 bl.length()=186 2024-03-04T18:21:51.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.149+0000 7f6410ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+ZFEULGCABYJCYFVXLEKB info.flags=0x7 2024-03-04T18:21:51.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.149+0000 7f6410ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+ZFEULGCABYJCYFVXLEKB to cache LRU end 2024-03-04T18:21:51.153 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-04T18:21:51.153 INFO:teuthology.orchestra.run.smithi137.stdout: "user_id": "foo.client.0", 2024-03-04T18:21:51.153 INFO:teuthology.orchestra.run.smithi137.stdout: "display_name": "Mr. foo.client.0", 2024-03-04T18:21:51.153 INFO:teuthology.orchestra.run.smithi137.stdout: "email": "foo.client.0+test@test.test", 2024-03-04T18:21:51.153 INFO:teuthology.orchestra.run.smithi137.stdout: "suspended": 0, 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: "max_buckets": 1000, 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: "subusers": [], 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: "keys": [ 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: "user": "foo.client.0", 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: "access_key": "ZFEULGCABYJCYFVXLEKB", 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: "secret_key": "eMKpNUy0MXcPyR7gOZ9H/4fIp/P5Ao/1KjEBKBTbuH//3d+Uet80kw==", 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: "active": true 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: "swift_keys": [], 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": [ 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "user-policy", 2024-03-04T18:21:51.154 INFO:teuthology.orchestra.run.smithi137.stdout: "perm": "*" 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: "op_mask": "read, write, delete", 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: "default_placement": "", 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: "default_storage_class": "", 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: "placement_tags": [], 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: "bucket_quota": { 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: "user_quota": { 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T18:21:51.155 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T18:21:51.156 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T18:21:51.156 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T18:21:51.156 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T18:21:51.156 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:51.156 INFO:teuthology.orchestra.run.smithi137.stdout: "temp_url_keys": [], 2024-03-04T18:21:51.156 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "rgw", 2024-03-04T18:21:51.156 INFO:teuthology.orchestra.run.smithi137.stdout: "mfa_ids": [] 2024-03-04T18:21:51.156 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-04T18:21:51.156 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:51.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.153+0000 7f63df7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:51.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.157+0000 7f653008d980 20 remove_watcher() i=0 2024-03-04T18:21:51.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.157+0000 7f653008d980 2 removed watcher, disabling cache 2024-03-04T18:21:51.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.161+0000 7f653008d980 20 remove_watcher() i=1 2024-03-04T18:21:51.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.161+0000 7f653008d980 20 remove_watcher() i=2 2024-03-04T18:21:51.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.165+0000 7f653008d980 20 remove_watcher() i=3 2024-03-04T18:21:51.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.165+0000 7f653008d980 20 remove_watcher() i=4 2024-03-04T18:21:51.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.169+0000 7f653008d980 20 remove_watcher() i=5 2024-03-04T18:21:51.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.173+0000 7f653008d980 20 remove_watcher() i=6 2024-03-04T18:21:51.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.173+0000 7f653008d980 20 remove_watcher() i=7 2024-03-04T18:21:51.192 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 4831902562 --totp-seed YR2ODWKMDZE7QSAV6LTQVXRCVIICYJSMMUUQN4DJRC6FMEVQMXIDMX23IRAUTOOK --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-03-04T18:21:51.231 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:21:51.232 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:21:51.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.269+0000 7f2fd4d92980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.269+0000 7f2fd4d92980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.269+0000 7f2fd4d92980 20 realm 2024-03-04T18:21:51.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.269+0000 7f2fd4d92980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.269+0000 7f2fd4d92980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:51.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.269+0000 7f2fd4d92980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:21:51.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.269+0000 7f2fd4d92980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.269+0000 7f2fd4d92980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.269+0000 7f2fd4d92980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.273+0000 7f2fd4d92980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.273+0000 7f2fd4d92980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.273+0000 7f2fd4d92980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:21:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.273+0000 7f2fd4d92980 20 searching for the correct realm 2024-03-04T18:21:51.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.277+0000 7f2fd30ab640 0 -- 172.21.15.137:0/1391513370 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f2fc4008b70 msgr2=0x7f2fc4026900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:51.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.493+0000 7f2fd30ab640 0 -- 172.21.15.137:0/1391513370 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55b38e7d6a90 msgr2=0x55b38e7d8f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:51.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.501+0000 7f2fd4d92980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:21:51.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.501+0000 7f2fd4d92980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:21:51.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.501+0000 7f2fd4d92980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:21:51.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.501+0000 7f2fd4d92980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:21:51.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.501+0000 7f2fd4d92980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:21:51.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.501+0000 7f2fd4d92980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:21:51.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.501+0000 7f2fd4d92980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.505+0000 7f2fd4d92980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:51.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.505+0000 7f2fd4d92980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.505+0000 7f2fd4d92980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:51.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.505+0000 7f2fd4d92980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.505+0000 7f2fd4d92980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:21:51.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.505+0000 7f2fd4d92980 20 zone default found 2024-03-04T18:21:51.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.505+0000 7f2fd4d92980 4 Realm: () 2024-03-04T18:21:51.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.505+0000 7f2fd4d92980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:21:51.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.505+0000 7f2fd4d92980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:21:51.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.505+0000 7f2fd4d92980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:21:51.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.505+0000 7f2fd4d92980 20 zonegroup default 2024-03-04T18:21:51.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.505+0000 7f2fd4d92980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.505+0000 7f2fd4d92980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:51.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.505+0000 7f2fd4d92980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.509+0000 7f2fd4d92980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:51.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.509+0000 7f2fd4d92980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.509+0000 7f2fd4d92980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:51.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.509+0000 7f2fd4d92980 20 started sync module instance, tier type = 2024-03-04T18:21:51.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.509+0000 7f2fd4d92980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:21:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.533+0000 7f2fd4d92980 20 add_watcher() i=0 2024-03-04T18:21:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.533+0000 7f2fd4d92980 20 add_watcher() i=1 2024-03-04T18:21:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.533+0000 7f2fd4d92980 20 add_watcher() i=2 2024-03-04T18:21:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.533+0000 7f2fd4d92980 20 add_watcher() i=3 2024-03-04T18:21:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.533+0000 7f2fd4d92980 20 add_watcher() i=4 2024-03-04T18:21:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.533+0000 7f2fd4d92980 20 add_watcher() i=5 2024-03-04T18:21:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.533+0000 7f2fd4d92980 20 add_watcher() i=6 2024-03-04T18:21:51.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.533+0000 7f2fd4d92980 20 add_watcher() i=7 2024-03-04T18:21:51.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.533+0000 7f2fd4d92980 2 all 8 watchers are set, enabling cache 2024-03-04T18:21:51.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.541+0000 7f2e89fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:51.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.541+0000 7f2fd4d92980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:21:51.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.541+0000 7f2fd4d92980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:21:51.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.541+0000 7f2fd4d92980 5 note: GC not initialized 2024-03-04T18:21:51.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.541+0000 7f2e88fe1640 20 reqs_thread_entry: start 2024-03-04T18:21:51.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.541+0000 7f2fd4d92980 20 init_complete bucket index max shards: 1 2024-03-04T18:21:51.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.541+0000 7f2fd4d92980 20 Filter name: none 2024-03-04T18:21:51.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.541+0000 7f2e81ffb640 20 reqs_thread_entry: start 2024-03-04T18:21:51.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.541+0000 7f2fd4d92980 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2024-03-04T18:21:51.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.541+0000 7f2fd4d92980 20 rados->read ofs=0 len=0 2024-03-04T18:21:51.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.541+0000 7f2fd4d92980 20 rados_obj.operate() r=0 bl.length=456 2024-03-04T18:21:51.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.541+0000 7f2fd4d92980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x13 2024-03-04T18:21:51.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.541+0000 7f2fd4d92980 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-03-04T18:21:51.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:51.541+0000 7f2fd4d92980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:21:52.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.705+0000 7f2fd4d92980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-03-04T18:21:52.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.705+0000 7f2fd4d92980 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-03-04T18:21:52.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.705+0000 7f2fd4d92980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2024-03-04T18:21:52.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.705+0000 7f2eb5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562508 cookie 94229678099808 notifier 7599 bl.length()=655 2024-03-04T18:21:52.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.705+0000 7f2eb5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-03-04T18:21:52.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.705+0000 7f2eb5ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-03-04T18:21:52.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.713+0000 7f2e89fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:52.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.717+0000 7f2fd4d92980 20 remove_watcher() i=0 2024-03-04T18:21:52.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.717+0000 7f2fd4d92980 2 removed watcher, disabling cache 2024-03-04T18:21:52.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.721+0000 7f2fd4d92980 20 remove_watcher() i=1 2024-03-04T18:21:52.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.721+0000 7f2fd4d92980 20 remove_watcher() i=2 2024-03-04T18:21:52.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.725+0000 7f2fd4d92980 20 remove_watcher() i=3 2024-03-04T18:21:52.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.729+0000 7f2fd4d92980 20 remove_watcher() i=4 2024-03-04T18:21:52.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.729+0000 7f2fd4d92980 20 remove_watcher() i=5 2024-03-04T18:21:52.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.733+0000 7f2fd4d92980 20 remove_watcher() i=6 2024-03-04T18:21:52.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.737+0000 7f2fd4d92980 20 remove_watcher() i=7 2024-03-04T18:21:52.757 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2024-03-04T18:21:52.757 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name 'Mr. bar.client.0' --email bar.client.0+test@test.test --caps 'user-policy=*' --access-key WKYNKKGETUBQGAHOULUX --secret DmSjI1Dt22vemZc5kXL+V8KMtEEzDT1KQGNkUp2b9FESPNapyaHixQ== --cluster ceph 2024-03-04T18:21:52.847 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:21:52.847 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:21:52.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.885+0000 7ff7a8794980 20 rados->read ofs=0 len=0 2024-03-04T18:21:52.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.889+0000 7ff7a8794980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:52.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.889+0000 7ff7a8794980 20 realm 2024-03-04T18:21:52.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.889+0000 7ff7a8794980 20 rados->read ofs=0 len=0 2024-03-04T18:21:52.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.889+0000 7ff7a8794980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:52.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.889+0000 7ff7a8794980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:21:52.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.889+0000 7ff7a8794980 20 rados->read ofs=0 len=0 2024-03-04T18:21:52.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.889+0000 7ff7a8794980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.889+0000 7ff7a8794980 20 rados->read ofs=0 len=0 2024-03-04T18:21:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.889+0000 7ff7a8794980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.889+0000 7ff7a8794980 20 rados->read ofs=0 len=0 2024-03-04T18:21:52.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.889+0000 7ff7a8794980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:21:52.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.889+0000 7ff7a8794980 20 searching for the correct realm 2024-03-04T18:21:52.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:52.897+0000 7ff7a6aad640 0 -- 172.21.15.137:0/3117806158 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7ff7a0059300 msgr2=0x7ff7a0016710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:53.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.113+0000 7ff7a6aad640 0 -- 172.21.15.137:0/3117806158 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x5589f5c4dc80 msgr2=0x5589f5c50140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:53.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.121+0000 7ff7a8794980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:21:53.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.121+0000 7ff7a8794980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:21:53.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.121+0000 7ff7a8794980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:21:53.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.121+0000 7ff7a8794980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:21:53.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.121+0000 7ff7a8794980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:21:53.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.121+0000 7ff7a8794980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:21:53.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.121+0000 7ff7a8794980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.121+0000 7ff7a8794980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.121+0000 7ff7a8794980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.121+0000 7ff7a8794980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:53.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.121+0000 7ff7a8794980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:21:53.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 20 zone default found 2024-03-04T18:21:53.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 4 Realm: () 2024-03-04T18:21:53.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:21:53.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:21:53.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:21:53.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 20 zonegroup default 2024-03-04T18:21:53.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 20 started sync module instance, tier type = 2024-03-04T18:21:53.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.125+0000 7ff7a8794980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:21:53.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.145+0000 7ff7a8794980 20 add_watcher() i=0 2024-03-04T18:21:53.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.145+0000 7ff7a8794980 20 add_watcher() i=1 2024-03-04T18:21:53.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.145+0000 7ff7a8794980 20 add_watcher() i=2 2024-03-04T18:21:53.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.145+0000 7ff7a8794980 20 add_watcher() i=3 2024-03-04T18:21:53.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.145+0000 7ff7a8794980 20 add_watcher() i=4 2024-03-04T18:21:53.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.145+0000 7ff7a8794980 20 add_watcher() i=5 2024-03-04T18:21:53.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.145+0000 7ff7a8794980 20 add_watcher() i=6 2024-03-04T18:21:53.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.149+0000 7ff7a8794980 20 add_watcher() i=7 2024-03-04T18:21:53.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.149+0000 7ff7a8794980 2 all 8 watchers are set, enabling cache 2024-03-04T18:21:53.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.153+0000 7ff65cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:53.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.153+0000 7ff7a8794980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:21:53.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.153+0000 7ff7a8794980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:21:53.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.153+0000 7ff7a8794980 5 note: GC not initialized 2024-03-04T18:21:53.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.153+0000 7ff5d77fe640 20 reqs_thread_entry: start 2024-03-04T18:21:53.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.153+0000 7ff7a8794980 20 init_complete bucket index max shards: 1 2024-03-04T18:21:53.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.153+0000 7ff7a8794980 20 Filter name: none 2024-03-04T18:21:53.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.153+0000 7ff5d57fa640 20 reqs_thread_entry: start 2024-03-04T18:21:53.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.153+0000 7ff7a8794980 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2024-03-04T18:21:53.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.153+0000 7ff7a8794980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2024-03-04T18:21:53.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-03-04T18:21:53.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2024-03-04T18:21:53.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2024-03-04T18:21:53.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2024-03-04T18:21:53.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 10 cache get: name=default.rgw.meta+users.keys+WKYNKKGETUBQGAHOULUX : miss 2024-03-04T18:21:53.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 10 cache put: name=default.rgw.meta+users.keys+WKYNKKGETUBQGAHOULUX info.flags=0x0 2024-03-04T18:21:53.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 10 adding default.rgw.meta+users.keys+WKYNKKGETUBQGAHOULUX to cache LRU end 2024-03-04T18:21:53.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 10 cache get: name=default.rgw.meta+users.keys+WKYNKKGETUBQGAHOULUX : hit (negative entry) 2024-03-04T18:21:53.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 10 cache get: name=default.rgw.meta+users.keys+WKYNKKGETUBQGAHOULUX : hit (negative entry) 2024-03-04T18:21:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-03-04T18:21:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-03-04T18:21:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.157+0000 7ff7a8794980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2024-03-04T18:21:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.161+0000 7ff689ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562582 cookie 94051022213024 notifier 7614 bl.length()=641 2024-03-04T18:21:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.161+0000 7ff689ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-03-04T18:21:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.161+0000 7ff689ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-03-04T18:21:53.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.165+0000 7ff7a8794980 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2024-03-04T18:21:53.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.165+0000 7ff7a8794980 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2024-03-04T18:21:53.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.165+0000 7ff7a8794980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:bar.client.0+test@test.test, ofs0, ns] 2024-03-04T18:21:53.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.165+0000 7ff689ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562583 cookie 94051022213024 notifier 7614 bl.length()=194 2024-03-04T18:21:53.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.165+0000 7ff689ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2024-03-04T18:21:53.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.165+0000 7ff689ffb640 10 rgw watcher librados: moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2024-03-04T18:21:53.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.169+0000 7ff7a8794980 10 cache put: name=default.rgw.meta+users.keys+WKYNKKGETUBQGAHOULUX info.flags=0x7 2024-03-04T18:21:53.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.169+0000 7ff7a8794980 10 moving default.rgw.meta+users.keys+WKYNKKGETUBQGAHOULUX to cache LRU end 2024-03-04T18:21:53.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.169+0000 7ff7a8794980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:WKYNKKGETUBQGAHOULUX, ofs0, ns] 2024-03-04T18:21:53.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.169+0000 7ff6897fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562456 cookie 94051022214080 notifier 7614 bl.length()=186 2024-03-04T18:21:53.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.169+0000 7ff6897fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+WKYNKKGETUBQGAHOULUX info.flags=0x7 2024-03-04T18:21:53.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.169+0000 7ff6897fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+WKYNKKGETUBQGAHOULUX to cache LRU end 2024-03-04T18:21:53.174 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: "user_id": "bar.client.0", 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: "display_name": "Mr. bar.client.0", 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: "email": "bar.client.0+test@test.test", 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: "suspended": 0, 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: "max_buckets": 1000, 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: "subusers": [], 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: "keys": [ 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: "user": "bar.client.0", 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: "access_key": "WKYNKKGETUBQGAHOULUX", 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: "secret_key": "DmSjI1Dt22vemZc5kXL+V8KMtEEzDT1KQGNkUp2b9FESPNapyaHixQ==", 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: "active": true 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: "swift_keys": [], 2024-03-04T18:21:53.175 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": [ 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "user-policy", 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: "perm": "*" 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: "op_mask": "read, write, delete", 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: "default_placement": "", 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: "default_storage_class": "", 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: "placement_tags": [], 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: "bucket_quota": { 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:53.176 INFO:teuthology.orchestra.run.smithi137.stdout: "user_quota": { 2024-03-04T18:21:53.177 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T18:21:53.177 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T18:21:53.177 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T18:21:53.177 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T18:21:53.177 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T18:21:53.177 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:53.177 INFO:teuthology.orchestra.run.smithi137.stdout: "temp_url_keys": [], 2024-03-04T18:21:53.177 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "rgw", 2024-03-04T18:21:53.177 INFO:teuthology.orchestra.run.smithi137.stdout: "mfa_ids": [] 2024-03-04T18:21:53.177 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-04T18:21:53.177 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:53.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.173+0000 7ff65cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:53.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.177+0000 7ff7a8794980 20 remove_watcher() i=0 2024-03-04T18:21:53.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.177+0000 7ff7a8794980 2 removed watcher, disabling cache 2024-03-04T18:21:53.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.181+0000 7ff7a8794980 20 remove_watcher() i=1 2024-03-04T18:21:53.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.181+0000 7ff7a8794980 20 remove_watcher() i=2 2024-03-04T18:21:53.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.185+0000 7ff7a8794980 20 remove_watcher() i=3 2024-03-04T18:21:53.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.185+0000 7ff7a8794980 20 remove_watcher() i=4 2024-03-04T18:21:53.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.189+0000 7ff7a8794980 20 remove_watcher() i=5 2024-03-04T18:21:53.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.193+0000 7ff7a8794980 20 remove_watcher() i=6 2024-03-04T18:21:53.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.193+0000 7ff7a8794980 20 remove_watcher() i=7 2024-03-04T18:21:53.214 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 1324242225 --totp-seed VINAMMETZ6A3RPWXJUD44DHSQGU5MVLWOJGYXT5FOWBMM4NZ2BIJRR5TL6YJFH37 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-03-04T18:21:53.257 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:21:53.257 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:21:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.293+0000 7f23291cc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.297+0000 7f23291cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.297+0000 7f23291cc980 20 realm 2024-03-04T18:21:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.297+0000 7f23291cc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.297+0000 7f23291cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.297+0000 7f23291cc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:21:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.297+0000 7f23291cc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.297+0000 7f23291cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.297+0000 7f23291cc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.297+0000 7f23291cc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.297+0000 7f23291cc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.297+0000 7f23291cc980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:21:53.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.297+0000 7f23291cc980 20 searching for the correct realm 2024-03-04T18:21:53.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.509+0000 7f23264e3640 0 -- 172.21.15.137:0/2742191785 >> [v2:172.21.15.137:6800/3095622620,v1:172.21.15.137:6801/3095622620] conn(0x7f23180d4940 msgr2=0x7f23180ec410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:53.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.761+0000 7f23264e3640 0 -- 172.21.15.137:0/2742191785 >> [v2:172.21.15.137:6809/3854517010,v1:172.21.15.137:6811/3854517010] conn(0x56268f773ff0 msgr2=0x56268f7763e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:53.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.777+0000 7f23291cc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:21:53.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.777+0000 7f23291cc980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:21:53.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.777+0000 7f23291cc980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:21:53.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.777+0000 7f23291cc980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:21:53.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.777+0000 7f23291cc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:21:53.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.777+0000 7f23291cc980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:21:53.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.777+0000 7f23291cc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.777+0000 7f23291cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.777+0000 7f23291cc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.777+0000 7f23291cc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:53.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.777+0000 7f23291cc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:21:53.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 20 zone default found 2024-03-04T18:21:53.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 4 Realm: () 2024-03-04T18:21:53.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:21:53.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:21:53.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:21:53.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 20 zonegroup default 2024-03-04T18:21:53.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 20 started sync module instance, tier type = 2024-03-04T18:21:53.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.781+0000 7f23291cc980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:21:53.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.805+0000 7f23291cc980 20 add_watcher() i=0 2024-03-04T18:21:53.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.805+0000 7f23291cc980 20 add_watcher() i=1 2024-03-04T18:21:53.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.805+0000 7f23291cc980 20 add_watcher() i=2 2024-03-04T18:21:53.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.805+0000 7f23291cc980 20 add_watcher() i=3 2024-03-04T18:21:53.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.805+0000 7f23291cc980 20 add_watcher() i=4 2024-03-04T18:21:53.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.805+0000 7f23291cc980 20 add_watcher() i=5 2024-03-04T18:21:53.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.805+0000 7f23291cc980 20 add_watcher() i=6 2024-03-04T18:21:53.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.809+0000 7f23291cc980 20 add_watcher() i=7 2024-03-04T18:21:53.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.809+0000 7f23291cc980 2 all 8 watchers are set, enabling cache 2024-03-04T18:21:53.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.813+0000 7f21dd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:53.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.813+0000 7f23291cc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:21:53.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.813+0000 7f23291cc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:21:53.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.813+0000 7f23291cc980 5 note: GC not initialized 2024-03-04T18:21:53.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.813+0000 7f2157fff640 20 reqs_thread_entry: start 2024-03-04T18:21:53.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.813+0000 7f23291cc980 20 init_complete bucket index max shards: 1 2024-03-04T18:21:53.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.813+0000 7f23291cc980 20 Filter name: none 2024-03-04T18:21:53.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.813+0000 7f2155ffb640 20 reqs_thread_entry: start 2024-03-04T18:21:53.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.813+0000 7f23291cc980 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2024-03-04T18:21:53.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.813+0000 7f23291cc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.813+0000 7f23291cc980 20 rados_obj.operate() r=0 bl.length=456 2024-03-04T18:21:53.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.813+0000 7f23291cc980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x13 2024-03-04T18:21:53.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.813+0000 7f23291cc980 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-03-04T18:21:53.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.813+0000 7f23291cc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:21:53.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.821+0000 7f23291cc980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-03-04T18:21:53.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.821+0000 7f23291cc980 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-03-04T18:21:53.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.821+0000 7f23291cc980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2024-03-04T18:21:53.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.821+0000 7f220a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529880 cookie 94723615736784 notifier 7193 bl.length()=655 2024-03-04T18:21:53.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.821+0000 7f220a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-03-04T18:21:53.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.821+0000 7f220a7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-03-04T18:21:53.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.825+0000 7f21dd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:53.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.829+0000 7f23291cc980 20 remove_watcher() i=0 2024-03-04T18:21:53.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.829+0000 7f23291cc980 2 removed watcher, disabling cache 2024-03-04T18:21:53.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.829+0000 7f23291cc980 20 remove_watcher() i=1 2024-03-04T18:21:53.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.833+0000 7f23291cc980 20 remove_watcher() i=2 2024-03-04T18:21:53.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.837+0000 7f23291cc980 20 remove_watcher() i=3 2024-03-04T18:21:53.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.837+0000 7f23291cc980 20 remove_watcher() i=4 2024-03-04T18:21:53.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.841+0000 7f23291cc980 20 remove_watcher() i=5 2024-03-04T18:21:53.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.841+0000 7f23291cc980 20 remove_watcher() i=6 2024-03-04T18:21:53.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.845+0000 7f23291cc980 20 remove_watcher() i=7 2024-03-04T18:21:53.864 DEBUG:tasks.s3tests:Creating user testx$tenanteduser.client.0 on client.0 2024-03-04T18:21:53.864 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name 'Mr. testx$tenanteduser.client.0' --email 'testx$tenanteduser.client.0+test@test.test' --caps 'user-policy=*' --access-key BYKQYVMITDGOSTVETKOM --secret fX4rxhQyX0cRUZtzw/T1sN0lHn/I8uJHkk4nGYugtpsbjP7jH0YagA== --cluster ceph 2024-03-04T18:21:53.906 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:21:53.906 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:21:53.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.941+0000 7fd7df426980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.945+0000 7fd7df426980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.945+0000 7fd7df426980 20 realm 2024-03-04T18:21:53.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.945+0000 7fd7df426980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.945+0000 7fd7df426980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.945+0000 7fd7df426980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:21:53.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.945+0000 7fd7df426980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.945+0000 7fd7df426980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:53.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.945+0000 7fd7df426980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.945+0000 7fd7df426980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:53.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.945+0000 7fd7df426980 20 rados->read ofs=0 len=0 2024-03-04T18:21:53.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.949+0000 7fd7df426980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:21:53.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.949+0000 7fd7df426980 20 searching for the correct realm 2024-03-04T18:21:53.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:53.953+0000 7fd7dd73f640 0 -- 172.21.15.137:0/4050293388 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fd7cc05c160 msgr2=0x7fd7cc076920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-03-04T18:21:54.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.165+0000 7fd7dd73f640 0 -- 172.21.15.137:0/4050293388 >> [v2:172.21.15.173:6806/2740311098,v1:172.21.15.173:6810/2740311098] conn(0x7fd6a80760f0 msgr2=0x7fd6a8078520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:54.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.177+0000 7fd7df426980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:21:54.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.177+0000 7fd7df426980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:21:54.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.177+0000 7fd7df426980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:21:54.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.177+0000 7fd7df426980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:21:54.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.177+0000 7fd7df426980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:21:54.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.177+0000 7fd7df426980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:21:54.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.177+0000 7fd7df426980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.177+0000 7fd7df426980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.177+0000 7fd7df426980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:54.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:21:54.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 20 zone default found 2024-03-04T18:21:54.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 4 Realm: () 2024-03-04T18:21:54.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:21:54.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:21:54.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:21:54.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 20 zonegroup default 2024-03-04T18:21:54.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 20 started sync module instance, tier type = 2024-03-04T18:21:54.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.181+0000 7fd7df426980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:21:54.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.205+0000 7fd7df426980 20 add_watcher() i=0 2024-03-04T18:21:54.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.205+0000 7fd7df426980 20 add_watcher() i=1 2024-03-04T18:21:54.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.205+0000 7fd7df426980 20 add_watcher() i=2 2024-03-04T18:21:54.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.205+0000 7fd7df426980 20 add_watcher() i=3 2024-03-04T18:21:54.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.205+0000 7fd7df426980 20 add_watcher() i=4 2024-03-04T18:21:54.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.205+0000 7fd7df426980 20 add_watcher() i=5 2024-03-04T18:21:54.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.205+0000 7fd7df426980 20 add_watcher() i=6 2024-03-04T18:21:54.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.209+0000 7fd7df426980 20 add_watcher() i=7 2024-03-04T18:21:54.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.209+0000 7fd7df426980 2 all 8 watchers are set, enabling cache 2024-03-04T18:21:54.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd6937e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:54.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:21:54.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:21:54.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 5 note: GC not initialized 2024-03-04T18:21:54.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd6927e4640 20 reqs_thread_entry: start 2024-03-04T18:21:54.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 20 init_complete bucket index max shards: 1 2024-03-04T18:21:54.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 20 Filter name: none 2024-03-04T18:21:54.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd68b7fe640 20 reqs_thread_entry: start 2024-03-04T18:21:54.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2024-03-04T18:21:54.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2024-03-04T18:21:54.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-03-04T18:21:54.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test : miss 2024-03-04T18:21:54.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x0 2024-03-04T18:21:54.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 10 adding default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2024-03-04T18:21:54.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 10 cache get: name=default.rgw.meta+users.keys+BYKQYVMITDGOSTVETKOM : miss 2024-03-04T18:21:54.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.213+0000 7fd7df426980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.217+0000 7fd7df426980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.217+0000 7fd7df426980 10 cache put: name=default.rgw.meta+users.keys+BYKQYVMITDGOSTVETKOM info.flags=0x0 2024-03-04T18:21:54.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.217+0000 7fd7df426980 10 adding default.rgw.meta+users.keys+BYKQYVMITDGOSTVETKOM to cache LRU end 2024-03-04T18:21:54.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.217+0000 7fd7df426980 10 cache get: name=default.rgw.meta+users.keys+BYKQYVMITDGOSTVETKOM : hit (negative entry) 2024-03-04T18:21:54.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.217+0000 7fd7df426980 10 cache get: name=default.rgw.meta+users.keys+BYKQYVMITDGOSTVETKOM : hit (negative entry) 2024-03-04T18:21:54.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.217+0000 7fd7df426980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-03-04T18:21:54.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.217+0000 7fd7df426980 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-03-04T18:21:54.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.217+0000 7fd7df426980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2024-03-04T18:21:54.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.221+0000 7fd6b3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529880 cookie 94368987120256 notifier 7656 bl.length()=715 2024-03-04T18:21:54.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.221+0000 7fd6b3fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-03-04T18:21:54.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.221+0000 7fd6b3fff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-03-04T18:21:54.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.225+0000 7fd7df426980 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2024-03-04T18:21:54.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.225+0000 7fd7df426980 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2024-03-04T18:21:54.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.225+0000 7fd7df426980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.email:testx$tenanteduser.client.0+test@test.test, ofs0, ns] 2024-03-04T18:21:54.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.225+0000 7fd6b3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529881 cookie 94368987114160 notifier 7656 bl.length()=224 2024-03-04T18:21:54.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.225+0000 7fd6b3fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2024-03-04T18:21:54.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.225+0000 7fd6b3fff640 10 rgw watcher librados: moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2024-03-04T18:21:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.229+0000 7fd7df426980 10 cache put: name=default.rgw.meta+users.keys+BYKQYVMITDGOSTVETKOM info.flags=0x7 2024-03-04T18:21:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.229+0000 7fd7df426980 10 moving default.rgw.meta+users.keys+BYKQYVMITDGOSTVETKOM to cache LRU end 2024-03-04T18:21:54.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.229+0000 7fd7df426980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.keys:BYKQYVMITDGOSTVETKOM, ofs0, ns] 2024-03-04T18:21:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.229+0000 7fd6b3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529882 cookie 94368987123824 notifier 7656 bl.length()=201 2024-03-04T18:21:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.229+0000 7fd6b3fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+BYKQYVMITDGOSTVETKOM info.flags=0x7 2024-03-04T18:21:54.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.229+0000 7fd6b3fff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+BYKQYVMITDGOSTVETKOM to cache LRU end 2024-03-04T18:21:54.234 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-04T18:21:54.234 INFO:teuthology.orchestra.run.smithi137.stdout: "user_id": "testx$tenanteduser.client.0", 2024-03-04T18:21:54.234 INFO:teuthology.orchestra.run.smithi137.stdout: "display_name": "Mr. testx$tenanteduser.client.0", 2024-03-04T18:21:54.235 INFO:teuthology.orchestra.run.smithi137.stdout: "email": "testx$tenanteduser.client.0+test@test.test", 2024-03-04T18:21:54.235 INFO:teuthology.orchestra.run.smithi137.stdout: "suspended": 0, 2024-03-04T18:21:54.235 INFO:teuthology.orchestra.run.smithi137.stdout: "max_buckets": 1000, 2024-03-04T18:21:54.235 INFO:teuthology.orchestra.run.smithi137.stdout: "subusers": [], 2024-03-04T18:21:54.235 INFO:teuthology.orchestra.run.smithi137.stdout: "keys": [ 2024-03-04T18:21:54.235 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:54.235 INFO:teuthology.orchestra.run.smithi137.stdout: "user": "testx$tenanteduser.client.0", 2024-03-04T18:21:54.235 INFO:teuthology.orchestra.run.smithi137.stdout: "access_key": "BYKQYVMITDGOSTVETKOM", 2024-03-04T18:21:54.236 INFO:teuthology.orchestra.run.smithi137.stdout: "secret_key": "fX4rxhQyX0cRUZtzw/T1sN0lHn/I8uJHkk4nGYugtpsbjP7jH0YagA==", 2024-03-04T18:21:54.236 INFO:teuthology.orchestra.run.smithi137.stdout: "active": true 2024-03-04T18:21:54.236 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T18:21:54.236 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T18:21:54.236 INFO:teuthology.orchestra.run.smithi137.stdout: "swift_keys": [], 2024-03-04T18:21:54.236 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": [ 2024-03-04T18:21:54.236 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:54.236 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "user-policy", 2024-03-04T18:21:54.236 INFO:teuthology.orchestra.run.smithi137.stdout: "perm": "*" 2024-03-04T18:21:54.237 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T18:21:54.237 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T18:21:54.237 INFO:teuthology.orchestra.run.smithi137.stdout: "op_mask": "read, write, delete", 2024-03-04T18:21:54.237 INFO:teuthology.orchestra.run.smithi137.stdout: "default_placement": "", 2024-03-04T18:21:54.237 INFO:teuthology.orchestra.run.smithi137.stdout: "default_storage_class": "", 2024-03-04T18:21:54.237 INFO:teuthology.orchestra.run.smithi137.stdout: "placement_tags": [], 2024-03-04T18:21:54.237 INFO:teuthology.orchestra.run.smithi137.stdout: "bucket_quota": { 2024-03-04T18:21:54.237 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T18:21:54.237 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T18:21:54.237 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T18:21:54.237 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T18:21:54.237 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T18:21:54.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:54.238 INFO:teuthology.orchestra.run.smithi137.stdout: "user_quota": { 2024-03-04T18:21:54.238 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T18:21:54.238 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T18:21:54.238 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T18:21:54.238 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T18:21:54.238 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T18:21:54.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:54.238 INFO:teuthology.orchestra.run.smithi137.stdout: "temp_url_keys": [], 2024-03-04T18:21:54.238 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "rgw", 2024-03-04T18:21:54.238 INFO:teuthology.orchestra.run.smithi137.stdout: "mfa_ids": [] 2024-03-04T18:21:54.238 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-04T18:21:54.238 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:54.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.233+0000 7fd6937e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:54.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.237+0000 7fd7df426980 20 remove_watcher() i=0 2024-03-04T18:21:54.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.237+0000 7fd7df426980 2 removed watcher, disabling cache 2024-03-04T18:21:54.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.241+0000 7fd7df426980 20 remove_watcher() i=1 2024-03-04T18:21:54.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.241+0000 7fd7df426980 20 remove_watcher() i=2 2024-03-04T18:21:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.245+0000 7fd7df426980 20 remove_watcher() i=3 2024-03-04T18:21:54.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.245+0000 7fd7df426980 20 remove_watcher() i=4 2024-03-04T18:21:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.249+0000 7fd7df426980 20 remove_watcher() i=5 2024-03-04T18:21:54.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.249+0000 7fd7df426980 20 remove_watcher() i=6 2024-03-04T18:21:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.253+0000 7fd7df426980 20 remove_watcher() i=7 2024-03-04T18:21:54.273 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 1288637898 --totp-seed U6APHE67ZPFC4KS57XDDAJDQBV5UK2LTLYJ7UCV2Q2OXCP3VGKN7YTJZ3OA5746X --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-03-04T18:21:54.363 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:21:54.363 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:21:54.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.401+0000 7fbbb9629980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.405+0000 7fbbb9629980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.405+0000 7fbbb9629980 20 realm 2024-03-04T18:21:54.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.405+0000 7fbbb9629980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.405+0000 7fbbb9629980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.405+0000 7fbbb9629980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:21:54.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.405+0000 7fbbb9629980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.405+0000 7fbbb9629980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.405+0000 7fbbb9629980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.405+0000 7fbbb9629980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:54.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.405+0000 7fbbb9629980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.405+0000 7fbbb9629980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:21:54.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.405+0000 7fbbb9629980 20 searching for the correct realm 2024-03-04T18:21:54.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.413+0000 7fbbb7942640 0 -- 172.21.15.137:0/1010099598 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fbbb000d3b0 msgr2=0x7fbbb000d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:54.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.629+0000 7fbbb7942640 0 -- 172.21.15.137:0/1010099598 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55af3403a220 msgr2=0x55af3403c6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:54.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.637+0000 7fbbb9629980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:21:54.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.637+0000 7fbbb9629980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:21:54.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.637+0000 7fbbb9629980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:21:54.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.637+0000 7fbbb9629980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:21:54.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.637+0000 7fbbb9629980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:21:54.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.637+0000 7fbbb9629980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:21:54.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.637+0000 7fbbb9629980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.637+0000 7fbbb9629980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.637+0000 7fbbb9629980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.641+0000 7fbbb9629980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:54.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.641+0000 7fbbb9629980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.641+0000 7fbbb9629980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:21:54.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.641+0000 7fbbb9629980 20 zone default found 2024-03-04T18:21:54.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.641+0000 7fbbb9629980 4 Realm: () 2024-03-04T18:21:54.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.641+0000 7fbbb9629980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:21:54.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.641+0000 7fbbb9629980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:21:54.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.641+0000 7fbbb9629980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:21:54.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.641+0000 7fbbb9629980 20 zonegroup default 2024-03-04T18:21:54.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.641+0000 7fbbb9629980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.641+0000 7fbbb9629980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.641+0000 7fbbb9629980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.641+0000 7fbbb9629980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.641+0000 7fbbb9629980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.645+0000 7fbbb9629980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.645+0000 7fbbb9629980 20 started sync module instance, tier type = 2024-03-04T18:21:54.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.645+0000 7fbbb9629980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:21:54.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.665+0000 7fbbb9629980 20 add_watcher() i=0 2024-03-04T18:21:54.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.665+0000 7fbbb9629980 20 add_watcher() i=1 2024-03-04T18:21:54.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.665+0000 7fbbb9629980 20 add_watcher() i=2 2024-03-04T18:21:54.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.665+0000 7fbbb9629980 20 add_watcher() i=3 2024-03-04T18:21:54.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.665+0000 7fbbb9629980 20 add_watcher() i=4 2024-03-04T18:21:54.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.665+0000 7fbbb9629980 20 add_watcher() i=5 2024-03-04T18:21:54.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.665+0000 7fbbb9629980 20 add_watcher() i=6 2024-03-04T18:21:54.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.669+0000 7fbbb9629980 20 add_watcher() i=7 2024-03-04T18:21:54.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.669+0000 7fbbb9629980 2 all 8 watchers are set, enabling cache 2024-03-04T18:21:54.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.673+0000 7fba6dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:54.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.673+0000 7fbbb9629980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:21:54.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.673+0000 7fbbb9629980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:21:54.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.673+0000 7fbbb9629980 5 note: GC not initialized 2024-03-04T18:21:54.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.673+0000 7fba6cfe1640 20 reqs_thread_entry: start 2024-03-04T18:21:54.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.673+0000 7fbbb9629980 20 init_complete bucket index max shards: 1 2024-03-04T18:21:54.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.673+0000 7fbbb9629980 20 Filter name: none 2024-03-04T18:21:54.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.673+0000 7fb9e67fc640 20 reqs_thread_entry: start 2024-03-04T18:21:54.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.673+0000 7fbbb9629980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2024-03-04T18:21:54.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.673+0000 7fbbb9629980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.677+0000 7fbbb9629980 20 rados_obj.operate() r=0 bl.length=515 2024-03-04T18:21:54.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.677+0000 7fbbb9629980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x13 2024-03-04T18:21:54.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.677+0000 7fbbb9629980 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-03-04T18:21:54.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.677+0000 7fbbb9629980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:21:54.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.681+0000 7fbbb9629980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-03-04T18:21:54.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.681+0000 7fbbb9629980 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-03-04T18:21:54.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.681+0000 7fbbb9629980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2024-03-04T18:21:54.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.681+0000 7fba9affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529883 cookie 94210980316400 notifier 7677 bl.length()=729 2024-03-04T18:21:54.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.681+0000 7fba9affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-03-04T18:21:54.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.681+0000 7fba9affd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-03-04T18:21:54.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.689+0000 7fba6dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:54.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.693+0000 7fbbb9629980 20 remove_watcher() i=0 2024-03-04T18:21:54.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.693+0000 7fbbb9629980 2 removed watcher, disabling cache 2024-03-04T18:21:54.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.693+0000 7fbbb9629980 20 remove_watcher() i=1 2024-03-04T18:21:54.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.697+0000 7fbbb9629980 20 remove_watcher() i=2 2024-03-04T18:21:54.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.701+0000 7fbbb9629980 20 remove_watcher() i=3 2024-03-04T18:21:54.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.701+0000 7fbbb9629980 20 remove_watcher() i=4 2024-03-04T18:21:54.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.705+0000 7fbbb9629980 20 remove_watcher() i=5 2024-03-04T18:21:54.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.705+0000 7fbbb9629980 20 remove_watcher() i=6 2024-03-04T18:21:54.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.709+0000 7fbbb9629980 20 remove_watcher() i=7 2024-03-04T18:21:54.728 DEBUG:tasks.s3tests:Creating user foobar.client.0 on client.0 2024-03-04T18:21:54.729 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foobar.client.0 --display-name 'Mr. foobar.client.0' --email foobar.client.0+test@test.test --caps 'user-policy=*' --access-key WYVSVZZHBUERGFVSPGZF --secret jUrIqvEAVxgC6smOFnNFD1tygoZf8ui2pqs2wuMxg+wqbK/smA/1gA== --cluster ceph 2024-03-04T18:21:54.770 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:21:54.770 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:21:54.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.805+0000 7f01df8dc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.809+0000 7f01df8dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.809+0000 7f01df8dc980 20 realm 2024-03-04T18:21:54.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.809+0000 7f01df8dc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.809+0000 7f01df8dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.809+0000 7f01df8dc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:21:54.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.809+0000 7f01df8dc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.809+0000 7f01df8dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:54.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.809+0000 7f01df8dc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.809+0000 7f01df8dc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:54.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.809+0000 7f01df8dc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:54.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.813+0000 7f01df8dc980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:21:54.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.813+0000 7f01df8dc980 20 searching for the correct realm 2024-03-04T18:21:54.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:54.817+0000 7f01ddbf5640 0 -- 172.21.15.137:0/1793254241 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f01d802a420 msgr2=0x7f01d802a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:55.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.033+0000 7f01ddbf5640 0 -- 172.21.15.137:0/1793254241 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x563e13eb8a30 msgr2=0x563e13ebaef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:55.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.041+0000 7f01df8dc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:21:55.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.041+0000 7f01df8dc980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:21:55.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.041+0000 7f01df8dc980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:21:55.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.041+0000 7f01df8dc980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:21:55.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.041+0000 7f01df8dc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:21:55.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.041+0000 7f01df8dc980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:21:55.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.041+0000 7f01df8dc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.041+0000 7f01df8dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.041+0000 7f01df8dc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:55.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:21:55.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 20 zone default found 2024-03-04T18:21:55.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 4 Realm: () 2024-03-04T18:21:55.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:21:55.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:21:55.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:21:55.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 20 zonegroup default 2024-03-04T18:21:55.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 20 started sync module instance, tier type = 2024-03-04T18:21:55.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.045+0000 7f01df8dc980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:21:55.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.073+0000 7f01df8dc980 20 add_watcher() i=0 2024-03-04T18:21:55.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.073+0000 7f01df8dc980 20 add_watcher() i=1 2024-03-04T18:21:55.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.073+0000 7f01df8dc980 20 add_watcher() i=2 2024-03-04T18:21:55.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.073+0000 7f01df8dc980 20 add_watcher() i=3 2024-03-04T18:21:55.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.073+0000 7f01df8dc980 20 add_watcher() i=4 2024-03-04T18:21:55.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.073+0000 7f01df8dc980 20 add_watcher() i=5 2024-03-04T18:21:55.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.073+0000 7f01df8dc980 20 add_watcher() i=6 2024-03-04T18:21:55.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.073+0000 7f01df8dc980 20 add_watcher() i=7 2024-03-04T18:21:55.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.073+0000 7f01df8dc980 2 all 8 watchers are set, enabling cache 2024-03-04T18:21:55.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.077+0000 7f0093fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:55.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.077+0000 7f01df8dc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:21:55.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.077+0000 7f01df8dc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:21:55.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.077+0000 7f01df8dc980 5 note: GC not initialized 2024-03-04T18:21:55.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.077+0000 7f0092fe5640 20 reqs_thread_entry: start 2024-03-04T18:21:55.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.077+0000 7f01df8dc980 20 init_complete bucket index max shards: 1 2024-03-04T18:21:55.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.077+0000 7f01df8dc980 20 Filter name: none 2024-03-04T18:21:55.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f0090fe1640 20 reqs_thread_entry: start 2024-03-04T18:21:55.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-03-04T18:21:55.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x0 2024-03-04T18:21:55.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:21:55.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 10 cache get: name=default.rgw.meta+users.email+foobar.client.0+test@test.test : miss 2024-03-04T18:21:55.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 10 cache put: name=default.rgw.meta+users.email+foobar.client.0+test@test.test info.flags=0x0 2024-03-04T18:21:55.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 10 adding default.rgw.meta+users.email+foobar.client.0+test@test.test to cache LRU end 2024-03-04T18:21:55.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 10 cache get: name=default.rgw.meta+users.keys+WYVSVZZHBUERGFVSPGZF : miss 2024-03-04T18:21:55.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 10 cache put: name=default.rgw.meta+users.keys+WYVSVZZHBUERGFVSPGZF info.flags=0x0 2024-03-04T18:21:55.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 10 adding default.rgw.meta+users.keys+WYVSVZZHBUERGFVSPGZF to cache LRU end 2024-03-04T18:21:55.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 10 cache get: name=default.rgw.meta+users.keys+WYVSVZZHBUERGFVSPGZF : hit (negative entry) 2024-03-04T18:21:55.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.081+0000 7f01df8dc980 10 cache get: name=default.rgw.meta+users.keys+WYVSVZZHBUERGFVSPGZF : hit (negative entry) 2024-03-04T18:21:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.085+0000 7f01df8dc980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:21:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.085+0000 7f01df8dc980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:21:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.085+0000 7f01df8dc980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-03-04T18:21:55.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.085+0000 7f00c0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529884 cookie 94824622193120 notifier 7698 bl.length()=656 2024-03-04T18:21:55.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.085+0000 7f00c0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:21:55.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.085+0000 7f00c0ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:21:55.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.093+0000 7f01df8dc980 10 cache put: name=default.rgw.meta+users.email+foobar.client.0+test@test.test info.flags=0x7 2024-03-04T18:21:55.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.093+0000 7f01df8dc980 10 moving default.rgw.meta+users.email+foobar.client.0+test@test.test to cache LRU end 2024-03-04T18:21:55.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.093+0000 7f01df8dc980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:foobar.client.0+test@test.test, ofs0, ns] 2024-03-04T18:21:55.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.093+0000 7f00c0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529885 cookie 94824622187872 notifier 7698 bl.length()=200 2024-03-04T18:21:55.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.093+0000 7f00c0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foobar.client.0+test@test.test info.flags=0x7 2024-03-04T18:21:55.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.093+0000 7f00c0ff9640 10 rgw watcher librados: moving default.rgw.meta+users.email+foobar.client.0+test@test.test to cache LRU end 2024-03-04T18:21:55.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.097+0000 7f01df8dc980 10 cache put: name=default.rgw.meta+users.keys+WYVSVZZHBUERGFVSPGZF info.flags=0x7 2024-03-04T18:21:55.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.097+0000 7f01df8dc980 10 moving default.rgw.meta+users.keys+WYVSVZZHBUERGFVSPGZF to cache LRU end 2024-03-04T18:21:55.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.097+0000 7f01df8dc980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.keys:WYVSVZZHBUERGFVSPGZF, ofs0, ns] 2024-03-04T18:21:55.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.101+0000 7f00c0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529886 cookie 94824622189408 notifier 7698 bl.length()=189 2024-03-04T18:21:55.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.101+0000 7f00c0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+WYVSVZZHBUERGFVSPGZF info.flags=0x7 2024-03-04T18:21:55.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.101+0000 7f00c0ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+WYVSVZZHBUERGFVSPGZF to cache LRU end 2024-03-04T18:21:55.104 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-04T18:21:55.104 INFO:teuthology.orchestra.run.smithi137.stdout: "user_id": "foobar.client.0", 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: "display_name": "Mr. foobar.client.0", 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: "email": "foobar.client.0+test@test.test", 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: "suspended": 0, 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: "max_buckets": 1000, 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: "subusers": [], 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: "keys": [ 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: "user": "foobar.client.0", 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: "access_key": "WYVSVZZHBUERGFVSPGZF", 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: "secret_key": "jUrIqvEAVxgC6smOFnNFD1tygoZf8ui2pqs2wuMxg+wqbK/smA/1gA==", 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: "active": true 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: "swift_keys": [], 2024-03-04T18:21:55.105 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": [ 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "user-policy", 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: "perm": "*" 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: "op_mask": "read, write, delete", 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: "default_placement": "", 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: "default_storage_class": "", 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: "placement_tags": [], 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: "bucket_quota": { 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T18:21:55.106 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T18:21:55.107 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:55.107 INFO:teuthology.orchestra.run.smithi137.stdout: "user_quota": { 2024-03-04T18:21:55.107 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T18:21:55.107 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T18:21:55.107 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T18:21:55.107 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T18:21:55.107 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T18:21:55.107 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:55.107 INFO:teuthology.orchestra.run.smithi137.stdout: "temp_url_keys": [], 2024-03-04T18:21:55.107 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "rgw", 2024-03-04T18:21:55.107 INFO:teuthology.orchestra.run.smithi137.stdout: "mfa_ids": [] 2024-03-04T18:21:55.107 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-04T18:21:55.107 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:55.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.105+0000 7f0093fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:55.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.109+0000 7f01df8dc980 20 remove_watcher() i=0 2024-03-04T18:21:55.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.109+0000 7f01df8dc980 2 removed watcher, disabling cache 2024-03-04T18:21:55.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.109+0000 7f01df8dc980 20 remove_watcher() i=1 2024-03-04T18:21:55.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.113+0000 7f01df8dc980 20 remove_watcher() i=2 2024-03-04T18:21:55.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.117+0000 7f01df8dc980 20 remove_watcher() i=3 2024-03-04T18:21:55.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.117+0000 7f01df8dc980 20 remove_watcher() i=4 2024-03-04T18:21:55.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.121+0000 7f01df8dc980 20 remove_watcher() i=5 2024-03-04T18:21:55.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.121+0000 7f01df8dc980 20 remove_watcher() i=6 2024-03-04T18:21:55.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.125+0000 7f01df8dc980 20 remove_watcher() i=7 2024-03-04T18:21:55.146 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foobar.client.0 --totp-serial 3165658897 --totp-seed N4EDLB2RYBLWX66OGS6B3MPSOADZE3KLV4OKCCI2FETAJYPOG3UQRAXRSJ3NZNGU --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-03-04T18:21:55.234 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:21:55.234 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:21:55.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.269+0000 7fce93900980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.273+0000 7fce93900980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.273+0000 7fce93900980 20 realm 2024-03-04T18:21:55.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.273+0000 7fce93900980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.273+0000 7fce93900980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.273+0000 7fce93900980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:21:55.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.273+0000 7fce93900980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.273+0000 7fce93900980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.273+0000 7fce93900980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.273+0000 7fce93900980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:55.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.273+0000 7fce93900980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.277+0000 7fce93900980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:21:55.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.277+0000 7fce93900980 20 searching for the correct realm 2024-03-04T18:21:55.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.281+0000 7fce91c19640 0 -- 172.21.15.137:0/1736913275 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fce8c0b0070 msgr2=0x7fce8c0bf400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:55.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.289+0000 7fce91418640 0 -- 172.21.15.137:0/1736913275 >> [v2:172.21.15.137:6824/1039931256,v1:172.21.15.137:6825/1039931256] conn(0x561cb1cd05a0 msgr2=0x561cb1cd29d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:55.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:21:55.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:21:55.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:21:55.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:21:55.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:21:55.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:21:55.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:55.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:21:55.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 zone default found 2024-03-04T18:21:55.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 4 Realm: () 2024-03-04T18:21:55.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:21:55.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:21:55.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:21:55.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 zonegroup default 2024-03-04T18:21:55.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 started sync module instance, tier type = 2024-03-04T18:21:55.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.297+0000 7fce93900980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:21:55.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.317+0000 7fce93900980 20 add_watcher() i=0 2024-03-04T18:21:55.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.317+0000 7fce93900980 20 add_watcher() i=1 2024-03-04T18:21:55.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.317+0000 7fce93900980 20 add_watcher() i=2 2024-03-04T18:21:55.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.317+0000 7fce93900980 20 add_watcher() i=3 2024-03-04T18:21:55.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.317+0000 7fce93900980 20 add_watcher() i=4 2024-03-04T18:21:55.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.317+0000 7fce93900980 20 add_watcher() i=5 2024-03-04T18:21:55.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.317+0000 7fce93900980 20 add_watcher() i=6 2024-03-04T18:21:55.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.321+0000 7fce93900980 20 add_watcher() i=7 2024-03-04T18:21:55.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.321+0000 7fce93900980 2 all 8 watchers are set, enabling cache 2024-03-04T18:21:55.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.321+0000 7fcd477e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:55.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.321+0000 7fce93900980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:21:55.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.321+0000 7fce93900980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:21:55.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.321+0000 7fce93900980 5 note: GC not initialized 2024-03-04T18:21:55.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.325+0000 7fcd467e4640 20 reqs_thread_entry: start 2024-03-04T18:21:55.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.325+0000 7fce93900980 20 init_complete bucket index max shards: 1 2024-03-04T18:21:55.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.325+0000 7fce93900980 20 Filter name: none 2024-03-04T18:21:55.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.325+0000 7fccb7fff640 20 reqs_thread_entry: start 2024-03-04T18:21:55.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.325+0000 7fce93900980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-03-04T18:21:55.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.325+0000 7fce93900980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.325+0000 7fce93900980 20 rados_obj.operate() r=0 bl.length=468 2024-03-04T18:21:55.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.325+0000 7fce93900980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-03-04T18:21:55.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.325+0000 7fce93900980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:21:55.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.325+0000 7fce93900980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:21:55.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.333+0000 7fce93900980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:21:55.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.333+0000 7fce93900980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:21:55.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.333+0000 7fce93900980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-03-04T18:21:55.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.333+0000 7fcd74ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529887 cookie 94681242137456 notifier 7713 bl.length()=670 2024-03-04T18:21:55.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.333+0000 7fcd74ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:21:55.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.333+0000 7fcd74ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:21:55.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.337+0000 7fcd477e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:55.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.341+0000 7fce93900980 20 remove_watcher() i=0 2024-03-04T18:21:55.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.341+0000 7fce93900980 2 removed watcher, disabling cache 2024-03-04T18:21:55.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.341+0000 7fce93900980 20 remove_watcher() i=1 2024-03-04T18:21:55.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.345+0000 7fce93900980 20 remove_watcher() i=2 2024-03-04T18:21:55.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.345+0000 7fce93900980 20 remove_watcher() i=3 2024-03-04T18:21:55.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.349+0000 7fce93900980 20 remove_watcher() i=4 2024-03-04T18:21:55.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.349+0000 7fce93900980 20 remove_watcher() i=5 2024-03-04T18:21:55.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.353+0000 7fce93900980 20 remove_watcher() i=6 2024-03-04T18:21:55.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.357+0000 7fce93900980 20 remove_watcher() i=7 2024-03-04T18:21:55.372 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 caps add --uid foobar.client.0 --caps 'roles=*' --cluster ceph 2024-03-04T18:21:55.404 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:21:55.404 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:21:55.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.441+0000 7f69f61a8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.445+0000 7f69f61a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.445+0000 7f69f61a8980 20 realm 2024-03-04T18:21:55.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.445+0000 7f69f61a8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.445+0000 7f69f61a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.445+0000 7f69f61a8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:21:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.445+0000 7f69f61a8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.445+0000 7f69f61a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.445+0000 7f69f61a8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.445+0000 7f69f61a8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.445+0000 7f69f61a8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.445+0000 7f69f61a8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:21:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.445+0000 7f69f61a8980 20 searching for the correct realm 2024-03-04T18:21:55.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.453+0000 7f69f44c1640 0 -- 172.21.15.137:0/1833102159 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7f69ec014030 msgr2=0x7f69ec07b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:55.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.669+0000 7f69f44c1640 0 -- 172.21.15.137:0/1833102159 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x55e0e8c6ac30 msgr2=0x55e0e8c6d0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:55.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:21:55.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:21:55.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:21:55.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:21:55.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:21:55.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:21:55.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:55.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:21:55.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 20 zone default found 2024-03-04T18:21:55.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 4 Realm: () 2024-03-04T18:21:55.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.677+0000 7f69f61a8980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:21:55.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.681+0000 7f69f61a8980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:21:55.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.681+0000 7f69f61a8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:21:55.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.681+0000 7f69f61a8980 20 zonegroup default 2024-03-04T18:21:55.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.681+0000 7f69f61a8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.681+0000 7f69f61a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.681+0000 7f69f61a8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.681+0000 7f69f61a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.681+0000 7f69f61a8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.681+0000 7f69f61a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.681+0000 7f69f61a8980 20 started sync module instance, tier type = 2024-03-04T18:21:55.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.681+0000 7f69f61a8980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:21:55.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.705+0000 7f69f61a8980 20 add_watcher() i=0 2024-03-04T18:21:55.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.705+0000 7f69f61a8980 20 add_watcher() i=1 2024-03-04T18:21:55.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.705+0000 7f69f61a8980 20 add_watcher() i=2 2024-03-04T18:21:55.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.705+0000 7f69f61a8980 20 add_watcher() i=3 2024-03-04T18:21:55.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.705+0000 7f69f61a8980 20 add_watcher() i=4 2024-03-04T18:21:55.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.705+0000 7f69f61a8980 20 add_watcher() i=5 2024-03-04T18:21:55.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.705+0000 7f69f61a8980 20 add_watcher() i=6 2024-03-04T18:21:55.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.709+0000 7f69f61a8980 20 add_watcher() i=7 2024-03-04T18:21:55.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.709+0000 7f69f61a8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:21:55.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.713+0000 7f68a9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:55.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.713+0000 7f69f61a8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:21:55.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.713+0000 7f69f61a8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:21:55.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.713+0000 7f69f61a8980 5 note: GC not initialized 2024-03-04T18:21:55.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.713+0000 7f68a8fe1640 20 reqs_thread_entry: start 2024-03-04T18:21:55.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.713+0000 7f69f61a8980 20 init_complete bucket index max shards: 1 2024-03-04T18:21:55.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.713+0000 7f69f61a8980 20 Filter name: none 2024-03-04T18:21:55.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.713+0000 7f68227fc640 20 reqs_thread_entry: start 2024-03-04T18:21:55.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.713+0000 7f69f61a8980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-03-04T18:21:55.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.713+0000 7f69f61a8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.717+0000 7f69f61a8980 20 rados_obj.operate() r=0 bl.length=482 2024-03-04T18:21:55.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.717+0000 7f69f61a8980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-03-04T18:21:55.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.717+0000 7f69f61a8980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:21:55.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.717+0000 7f69f61a8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:21:55.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.721+0000 7f69f61a8980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:21:55.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.721+0000 7f69f61a8980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:21:55.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.721+0000 7f69f61a8980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-03-04T18:21:55.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.721+0000 7f68d2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529888 cookie 94424466400160 notifier 7731 bl.length()=683 2024-03-04T18:21:55.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.721+0000 7f68d2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:21:55.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.721+0000 7f68d2ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:21:55.725 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-04T18:21:55.725 INFO:teuthology.orchestra.run.smithi137.stdout: "user_id": "foobar.client.0", 2024-03-04T18:21:55.725 INFO:teuthology.orchestra.run.smithi137.stdout: "display_name": "Mr. foobar.client.0", 2024-03-04T18:21:55.725 INFO:teuthology.orchestra.run.smithi137.stdout: "email": "foobar.client.0+test@test.test", 2024-03-04T18:21:55.725 INFO:teuthology.orchestra.run.smithi137.stdout: "suspended": 0, 2024-03-04T18:21:55.725 INFO:teuthology.orchestra.run.smithi137.stdout: "max_buckets": 1000, 2024-03-04T18:21:55.725 INFO:teuthology.orchestra.run.smithi137.stdout: "subusers": [], 2024-03-04T18:21:55.725 INFO:teuthology.orchestra.run.smithi137.stdout: "keys": [ 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: "user": "foobar.client.0", 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: "access_key": "WYVSVZZHBUERGFVSPGZF", 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: "secret_key": "jUrIqvEAVxgC6smOFnNFD1tygoZf8ui2pqs2wuMxg+wqbK/smA/1gA==", 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: "active": true 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: "swift_keys": [], 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": [ 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "roles", 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: "perm": "*" 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "user-policy", 2024-03-04T18:21:55.726 INFO:teuthology.orchestra.run.smithi137.stdout: "perm": "*" 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "op_mask": "read, write, delete", 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "default_placement": "", 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "default_storage_class": "", 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "placement_tags": [], 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "bucket_quota": { 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "user_quota": { 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T18:21:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T18:21:55.728 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T18:21:55.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:55.728 INFO:teuthology.orchestra.run.smithi137.stdout: "temp_url_keys": [], 2024-03-04T18:21:55.728 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "rgw", 2024-03-04T18:21:55.728 INFO:teuthology.orchestra.run.smithi137.stdout: "mfa_ids": [ 2024-03-04T18:21:55.728 INFO:teuthology.orchestra.run.smithi137.stdout: "3165658897" 2024-03-04T18:21:55.728 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-03-04T18:21:55.728 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-04T18:21:55.728 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:55.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.725+0000 7f68a9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:55.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.729+0000 7f69f61a8980 20 remove_watcher() i=0 2024-03-04T18:21:55.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.729+0000 7f69f61a8980 2 removed watcher, disabling cache 2024-03-04T18:21:55.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.729+0000 7f69f61a8980 20 remove_watcher() i=1 2024-03-04T18:21:55.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.733+0000 7f69f61a8980 20 remove_watcher() i=2 2024-03-04T18:21:55.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.733+0000 7f69f61a8980 20 remove_watcher() i=3 2024-03-04T18:21:55.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.737+0000 7f69f61a8980 20 remove_watcher() i=4 2024-03-04T18:21:55.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.741+0000 7f69f61a8980 20 remove_watcher() i=5 2024-03-04T18:21:55.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.741+0000 7f69f61a8980 20 remove_watcher() i=6 2024-03-04T18:21:55.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.745+0000 7f69f61a8980 20 remove_watcher() i=7 2024-03-04T18:21:55.764 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 caps add --uid foobar.client.0 --caps 'oidc-provider=*' --cluster ceph 2024-03-04T18:21:55.843 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T18:21:55.843 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T18:21:55.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.881+0000 7fda69bf8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.885+0000 7fda69bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.885+0000 7fda69bf8980 20 realm 2024-03-04T18:21:55.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.885+0000 7fda69bf8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.885+0000 7fda69bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.885+0000 7fda69bf8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-04T18:21:55.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.885+0000 7fda69bf8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.885+0000 7fda69bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:55.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.885+0000 7fda69bf8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.885+0000 7fda69bf8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:55.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.885+0000 7fda69bf8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:55.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.885+0000 7fda69bf8980 20 rados_obj.operate() r=0 bl.length=1021 2024-03-04T18:21:55.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.885+0000 7fda69bf8980 20 searching for the correct realm 2024-03-04T18:21:55.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:55.893+0000 7fda67f11640 0 -- 172.21.15.137:0/2324028316 >> [v2:172.21.15.173:6800/3687068977,v1:172.21.15.173:6801/3687068977] conn(0x7fda600cd450 msgr2=0x7fda600cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:56.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.109+0000 7fda67f11640 0 -- 172.21.15.137:0/2324028316 >> [v2:172.21.15.173:6809/4214690098,v1:172.21.15.173:6813/4214690098] conn(0x560309ad39f0 msgr2=0x560309ad5eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-03-04T18:21:56.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-04T18:21:56.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 RGWRados::pool_iterate: got zone_info.f85b0d2e-c665-46d3-ae77-23c29725ed4a 2024-03-04T18:21:56.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-04T18:21:56.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-04T18:21:56.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-04T18:21:56.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 RGWRados::pool_iterate: got zonegroup_info.ebce986a-0940-4ce2-8dbd-a2d2fbb158dc 2024-03-04T18:21:56.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:56.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:56.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:56.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-04T18:21:56.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 rados_obj.operate() r=0 bl.length=432 2024-03-04T18:21:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 zone default found 2024-03-04T18:21:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 4 Realm: () 2024-03-04T18:21:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 4 ZoneGroup: default (ebce986a-0940-4ce2-8dbd-a2d2fbb158dc) 2024-03-04T18:21:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 4 Zone: default (f85b0d2e-c665-46d3-ae77-23c29725ed4a) 2024-03-04T18:21:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-04T18:21:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 zonegroup default 2024-03-04T18:21:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.117+0000 7fda69bf8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.121+0000 7fda69bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.121+0000 7fda69bf8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:56.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.121+0000 7fda69bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:56.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.121+0000 7fda69bf8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:56.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.121+0000 7fda69bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-04T18:21:56.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.121+0000 7fda69bf8980 20 started sync module instance, tier type = 2024-03-04T18:21:56.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.121+0000 7fda69bf8980 20 started zone id=f85b0d2e-c665-46d3-ae77-23c29725ed4a (name=default) with tier type = 2024-03-04T18:21:56.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.145+0000 7fda69bf8980 20 add_watcher() i=0 2024-03-04T18:21:56.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.145+0000 7fda69bf8980 20 add_watcher() i=1 2024-03-04T18:21:56.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.145+0000 7fda69bf8980 20 add_watcher() i=2 2024-03-04T18:21:56.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.145+0000 7fda69bf8980 20 add_watcher() i=3 2024-03-04T18:21:56.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.145+0000 7fda69bf8980 20 add_watcher() i=4 2024-03-04T18:21:56.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.145+0000 7fda69bf8980 20 add_watcher() i=5 2024-03-04T18:21:56.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.145+0000 7fda69bf8980 20 add_watcher() i=6 2024-03-04T18:21:56.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.149+0000 7fda69bf8980 20 add_watcher() i=7 2024-03-04T18:21:56.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.149+0000 7fda69bf8980 2 all 8 watchers are set, enabling cache 2024-03-04T18:21:56.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.153+0000 7fd91dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:56.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.153+0000 7fda69bf8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-04T18:21:56.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.153+0000 7fda69bf8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-04T18:21:56.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.153+0000 7fda69bf8980 5 note: GC not initialized 2024-03-04T18:21:56.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.153+0000 7fd91cfe1640 20 reqs_thread_entry: start 2024-03-04T18:21:56.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.153+0000 7fda69bf8980 20 init_complete bucket index max shards: 1 2024-03-04T18:21:56.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.153+0000 7fda69bf8980 20 Filter name: none 2024-03-04T18:21:56.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.153+0000 7fd8967fc640 20 reqs_thread_entry: start 2024-03-04T18:21:56.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.153+0000 7fda69bf8980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-03-04T18:21:56.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.153+0000 7fda69bf8980 20 rados->read ofs=0 len=0 2024-03-04T18:21:56.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.153+0000 7fda69bf8980 20 rados_obj.operate() r=0 bl.length=495 2024-03-04T18:21:56.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.153+0000 7fda69bf8980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-03-04T18:21:56.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.153+0000 7fda69bf8980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:21:56.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.153+0000 7fda69bf8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-04T18:21:56.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.157+0000 7fda69bf8980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:21:56.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.157+0000 7fda69bf8980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:21:56.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.157+0000 7fda69bf8980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-03-04T18:21:56.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.161+0000 7fd94affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529889 cookie 94571047266816 notifier 7749 bl.length()=704 2024-03-04T18:21:56.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.161+0000 7fd94affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-03-04T18:21:56.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.161+0000 7fd94affd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-03-04T18:21:56.164 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-04T18:21:56.164 INFO:teuthology.orchestra.run.smithi137.stdout: "user_id": "foobar.client.0", 2024-03-04T18:21:56.164 INFO:teuthology.orchestra.run.smithi137.stdout: "display_name": "Mr. foobar.client.0", 2024-03-04T18:21:56.164 INFO:teuthology.orchestra.run.smithi137.stdout: "email": "foobar.client.0+test@test.test", 2024-03-04T18:21:56.164 INFO:teuthology.orchestra.run.smithi137.stdout: "suspended": 0, 2024-03-04T18:21:56.164 INFO:teuthology.orchestra.run.smithi137.stdout: "max_buckets": 1000, 2024-03-04T18:21:56.164 INFO:teuthology.orchestra.run.smithi137.stdout: "subusers": [], 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: "keys": [ 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: "user": "foobar.client.0", 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: "access_key": "WYVSVZZHBUERGFVSPGZF", 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: "secret_key": "jUrIqvEAVxgC6smOFnNFD1tygoZf8ui2pqs2wuMxg+wqbK/smA/1gA==", 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: "active": true 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: "swift_keys": [], 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": [ 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "oidc-provider", 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: "perm": "*" 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "roles", 2024-03-04T18:21:56.165 INFO:teuthology.orchestra.run.smithi137.stdout: "perm": "*" 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "user-policy", 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: "perm": "*" 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: "op_mask": "read, write, delete", 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: "default_placement": "", 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: "default_storage_class": "", 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: "placement_tags": [], 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: "bucket_quota": { 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T18:21:56.166 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: "user_quota": { 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": false, 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: "check_on_raw": false, 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size": -1, 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: "max_size_kb": 0, 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: "max_objects": -1 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: "temp_url_keys": [], 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "rgw", 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: "mfa_ids": [ 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: "3165658897" 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-04T18:21:56.167 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:21:56.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.165+0000 7fd91dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-04T18:21:56.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.169+0000 7fda69bf8980 20 remove_watcher() i=0 2024-03-04T18:21:56.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.169+0000 7fda69bf8980 2 removed watcher, disabling cache 2024-03-04T18:21:56.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.169+0000 7fda69bf8980 20 remove_watcher() i=1 2024-03-04T18:21:56.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.173+0000 7fda69bf8980 20 remove_watcher() i=2 2024-03-04T18:21:56.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.177+0000 7fda69bf8980 20 remove_watcher() i=3 2024-03-04T18:21:56.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.177+0000 7fda69bf8980 20 remove_watcher() i=4 2024-03-04T18:21:56.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.181+0000 7fda69bf8980 20 remove_watcher() i=5 2024-03-04T18:21:56.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.181+0000 7fda69bf8980 20 remove_watcher() i=6 2024-03-04T18:21:56.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-04T18:21:56.185+0000 7fda69bf8980 20 remove_watcher() i=7 2024-03-04T18:21:56.203 INFO:tasks.s3tests:Configuring s3-tests... 2024-03-04T18:21:56.203 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T18:21:56.203 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2024-03-04T18:21:56.250 INFO:tasks.s3tests:Configuring boto... 2024-03-04T18:21:56.250 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-04T18:21:56.251 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/boto-client.0.cfg 2024-03-04T18:21:56.297 DEBUG:teuthology.orchestra.run.smithi137:s3 tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' 2024-03-04T18:21:56.459 INFO:teuthology.orchestra.run.smithi137.stdout:GLOB sdist-make: /home/ubuntu/cephtest/s3-tests-client.0/setup.py 2024-03-04T18:21:56.653 INFO:teuthology.orchestra.run.smithi137.stdout:py create: /home/ubuntu/cephtest/s3-tests-client.0/.tox/py 2024-03-04T18:21:56.925 INFO:teuthology.orchestra.run.smithi137.stdout:py installdeps: -rrequirements.txt 2024-03-04T18:23:32.105 INFO:teuthology.orchestra.run.smithi137.stdout:py inst: /home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.zip 2024-03-04T18:23:34.074 INFO:teuthology.orchestra.run.smithi137.stdout:py installed: boto==2.49.0,boto3==1.24.96,botocore==1.27.96,cachetools==5.3.3,certifi==2024.2.2,chardet==5.2.0,charset-normalizer==3.3.2,colorama==0.4.6,distlib==0.3.8,exceptiongroup==1.2.0,filelock==3.13.1,gevent==24.2.1,greenlet==3.0.3,httplib2==0.22.0,idna==3.6,iniconfig==2.0.0,isodate==0.6.1,jmespath==1.0.1,lxml==5.1.0,munch==4.0.0,packaging==23.2,platformdirs==4.2.0,pluggy==1.4.0,pyparsing==3.1.1,pyproject-api==1.6.1,pytest==8.0.2,python-dateutil==2.9.0.post0,pytz==2024.1,PyYAML==6.0.1,requests==2.31.0,s3tests @ file:///home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.zip#sha256=54b5c21700812b56397a96b2793b634b10184029940805805dacb27304aa2e1c,s3transfer==0.6.2,six==1.16.0,tomli==2.0.1,tox==4.13.0,urllib3==1.26.18,virtualenv==20.25.1,zope.event==5.0,zope.interface==6.2 2024-03-04T18:23:34.074 INFO:teuthology.orchestra.run.smithi137.stdout:py run-test-pre: PYTHONHASHSEED='259212422' 2024-03-04T18:23:34.074 INFO:teuthology.orchestra.run.smithi137.stdout:py run-test: commands[0] | pytest -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' 2024-03-04T18:23:34.261 INFO:teuthology.orchestra.run.smithi137.stdout:============================= test session starts ============================== 2024-03-04T18:23:34.262 INFO:teuthology.orchestra.run.smithi137.stdout:platform linux -- Python 3.10.6, pytest-8.0.2, pluggy-1.4.0 -- /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/bin/python 2024-03-04T18:23:34.262 INFO:teuthology.orchestra.run.smithi137.stdout:cachedir: .tox/py/.pytest_cache 2024-03-04T18:23:34.262 INFO:teuthology.orchestra.run.smithi137.stdout:rootdir: /home/ubuntu/cephtest/s3-tests-client.0 2024-03-04T18:23:34.262 INFO:teuthology.orchestra.run.smithi137.stdout:configfile: pytest.ini 2024-03-04T18:23:35.798 INFO:teuthology.orchestra.run.smithi137.stdout:collecting ... collected 787 items / 124 deselected / 663 selected 2024-03-04T18:23:35.798 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-04T18:23:36.918 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_none PASSED [ 0%] 2024-03-04T18:23:37.542 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_empty PASSED [ 0%] 2024-03-04T18:23:38.122 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_date_and_amz_date PASSED [ 0%] 2024-03-04T18:23:38.728 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_amz_date_and_no_date PASSED [ 0%] 2024-03-04T18:23:39.342 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_none PASSED [ 0%] 2024-03-04T18:23:39.800 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_contentlength_none PASSED [ 0%] 2024-03-04T18:23:40.543 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_acl_create_contentlength_none PASSED [ 1%] 2024-03-04T18:23:41.004 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_contentlength_none PASSED [ 1%] 2024-03-04T18:23:41.182 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_empty PASSED [ 1%] 2024-03-04T18:23:41.359 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_none PASSED [ 1%] 2024-03-04T18:23:41.967 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_mismatch_below_aws2 PASSED [ 1%] 2024-03-04T18:23:42.592 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_incorrect_aws2 PASSED [ 1%] 2024-03-04T18:23:43.214 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_invalid_aws2 PASSED [ 1%] 2024-03-04T18:23:43.857 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_none_aws2 PASSED [ 2%] 2024-03-04T18:23:44.049 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_invalid_aws2 PASSED [ 2%] 2024-03-04T18:23:44.232 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_none_aws2 PASSED [ 2%] 2024-03-04T18:23:44.355 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws4 SKIPPED [ 2%] 2024-03-04T18:23:44.483 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_mismatch_below_aws4 SKIPPED [ 2%] 2024-03-04T18:23:44.615 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_incorrect_aws4 SKIPPED [ 2%] 2024-03-04T18:23:44.746 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_invalid_aws4 SKIPPED [ 3%] 2024-03-04T18:23:44.868 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_empty_aws4 SKIPPED [ 3%] 2024-03-04T18:23:44.996 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_none_aws4 SKIPPED [ 3%] 2024-03-04T18:23:45.131 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_invalid_aws4 SKIPPED [ 3%] 2024-03-04T18:23:45.257 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_invalid_aws4 SKIPPED [ 3%] 2024-03-04T18:23:45.388 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_empty_aws4 SKIPPED [ 3%] 2024-03-04T18:23:45.515 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_empty_aws4 SKIPPED [ 3%] 2024-03-04T18:23:45.643 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_none_aws4 SKIPPED [ 4%] 2024-03-04T18:23:45.771 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_none_aws4 SKIPPED [ 4%] 2024-03-04T18:23:45.900 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_today_aws4 SKIPPED [ 4%] 2024-03-04T18:23:46.028 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_before_today_aws4 SKIPPED [ 4%] 2024-03-04T18:23:46.159 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_today_aws4 SKIPPED [ 4%] 2024-03-04T18:23:46.290 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_after_today_aws4 SKIPPED [ 4%] 2024-03-04T18:23:46.416 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws4 SKIPPED [ 4%] 2024-03-04T18:23:46.543 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_before_epoch_aws4 SKIPPED [ 5%] 2024-03-04T18:23:46.670 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_end_aws4 SKIPPED [ 5%] 2024-03-04T18:23:46.820 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_after_end_aws4 SKIPPED [ 5%] 2024-03-04T18:23:46.947 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_missing_signed_custom_header_aws4 SKIPPED [ 5%] 2024-03-04T18:23:47.075 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_object_create_missing_signed_header_aws4 SKIPPED [ 5%] 2024-03-04T18:23:47.205 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_invalid_aws4 SKIPPED [ 5%] 2024-03-04T18:23:47.334 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_ua_empty_aws4 SKIPPED [ 6%] 2024-03-04T18:23:47.461 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_ua_none_aws4 SKIPPED [ 6%] 2024-03-04T18:23:47.588 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws4 SKIPPED [ 6%] 2024-03-04T18:23:47.715 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_invalid_aws4 SKIPPED [ 6%] 2024-03-04T18:23:47.843 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_empty_aws4 SKIPPED [ 6%] 2024-03-04T18:23:47.972 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_empty_aws4 SKIPPED [ 6%] 2024-03-04T18:23:48.099 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_none_aws4 SKIPPED [ 6%] 2024-03-04T18:23:48.231 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_none_aws4 SKIPPED [ 7%] 2024-03-04T18:23:48.374 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws4 SKIPPED [ 7%] 2024-03-04T18:23:48.502 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_today_aws4 SKIPPED [ 7%] 2024-03-04T18:23:48.632 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws4 SKIPPED [ 7%] 2024-03-04T18:23:48.762 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_after_today_aws4 SKIPPED [ 7%] 2024-03-04T18:23:48.891 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws4 SKIPPED [ 7%] 2024-03-04T18:23:49.019 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_epoch_aws4 SKIPPED [ 7%] 2024-03-04T18:23:50.429 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3.py::test_versioning_obj_read_not_exist_null PASSED [ 8%] 2024-03-04T18:24:09.227 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3.py::test_object_storage_class PASSED [ 8%] 2024-03-04T18:24:28.527 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3.py::test_object_storage_class_multipart PASSED [ 8%] 2024-03-04T18:24:59.072 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3.py::test_object_modify_storage_class PASSED [ 8%] 2024-03-04T18:25:36.664 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3.py::test_object_modify_storage_class_multipart PASSED [ 8%] 2024-03-04T18:25:46.961 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3.py::test_object_storage_class_copy PASSED [ 8%] 2024-03-04T18:25:58.385 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3.py::test_object_storage_class_copy_multipart PASSED [ 9%] 2024-03-04T18:26:01.578 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3.py::test_atomic_dual_conditional_write_1mb PASSED [ 9%] 2024-03-04T18:26:02.194 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3.py::test_atomic_write_bucket_gone PASSED [ 9%] 2024-03-04T18:26:06.862 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_invalid_chunks_1 PASSED [ 9%] 2024-03-04T18:26:11.873 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_invalid_chunks_2 PASSED [ 9%] 2024-03-04T18:26:14.002 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3.py::test_bucket_policy_set_condition_operator_end_with_IfExists PASSED [ 9%] 2024-03-04T18:26:15.261 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3.py::test_bucket_policy_put_obj_request_obj_tag PASSED [ 9%] 2024-03-04T18:26:16.168 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_nonexistant_bucket_rgw SKIPPED [ 10%] 2024-03-04T18:26:16.170 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_public_index SKIPPED [ 10%] 2024-03-04T18:26:16.176 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_public_index SKIPPED [ 10%] 2024-03-04T18:26:16.179 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty SKIPPED [ 10%] 2024-03-04T18:26:16.181 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty SKIPPED [ 10%] 2024-03-04T18:26:16.184 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index SKIPPED [ 10%] 2024-03-04T18:26:16.187 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index SKIPPED [ 11%] 2024-03-04T18:26:16.189 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_missingerrordoc SKIPPED [ 11%] 2024-03-04T18:26:16.192 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_missingerrordoc SKIPPED [ 11%] 2024-03-04T18:26:16.196 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_missingerrordoc SKIPPED [ 11%] 2024-03-04T18:26:16.200 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_missingerrordoc SKIPPED [ 11%] 2024-03-04T18:26:16.204 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_blockederrordoc SKIPPED [ 11%] 2024-03-04T18:26:16.206 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_pubilc_errordoc SKIPPED [ 11%] 2024-03-04T18:26:16.209 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_blockederrordoc SKIPPED [ 12%] 2024-03-04T18:26:16.213 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_blockederrordoc SKIPPED [ 12%] 2024-03-04T18:26:16.215 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_blockederrordoc SKIPPED [ 12%] 2024-03-04T18:26:16.218 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_gooderrordoc SKIPPED [ 12%] 2024-03-04T18:26:16.220 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_gooderrordoc SKIPPED [ 12%] 2024-03-04T18:26:16.223 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_gooderrordoc SKIPPED [ 12%] 2024-03-04T18:26:16.226 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_gooderrordoc SKIPPED [ 12%] 2024-03-04T18:26:16.228 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_base SKIPPED [ 13%] 2024-03-04T18:26:16.230 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_path SKIPPED [ 13%] 2024-03-04T18:26:16.232 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_path_upgrade SKIPPED [ 13%] 2024-03-04T18:26:16.234 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_nonwebsite SKIPPED [ 13%] 2024-03-04T18:26:16.237 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_public_relative SKIPPED [ 13%] 2024-03-04T18:26:16.239 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_public_abs SKIPPED [ 13%] 2024-03-04T18:26:16.241 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_private_relative SKIPPED [ 14%] 2024-03-04T18:26:16.243 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_private_abs SKIPPED [ 14%] 2024-03-04T18:26:16.343 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_s3_website.py::test_routing_generator XFAIL [ 14%] 2024-03-04T18:26:16.358 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests/functional/test_utils.py::test_generate PASSED [ 14%] 2024-03-04T18:26:18.447 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_invalid_short PASSED [ 14%] 2024-03-04T18:26:27.532 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_bad PASSED [ 14%] 2024-03-04T18:26:28.946 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_empty PASSED [ 14%] 2024-03-04T18:26:30.684 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_none PASSED [ 15%] 2024-03-04T18:26:32.406 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_mismatch PASSED [ 15%] 2024-03-04T18:26:34.087 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_empty PASSED [ 15%] 2024-03-04T18:26:35.822 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_none PASSED [ 15%] 2024-03-04T18:26:37.265 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contentlength_negative PASSED [ 15%] 2024-03-04T18:26:38.976 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_invalid PASSED [ 15%] 2024-03-04T18:26:40.530 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_empty PASSED [ 15%] 2024-03-04T18:26:42.046 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_none PASSED [ 16%] 2024-03-04T18:26:43.395 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_put_bad_canned_acl PASSED [ 16%] 2024-03-04T18:26:44.605 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_expect_mismatch PASSED [ 16%] 2024-03-04T18:26:45.841 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_expect_empty PASSED [ 16%] 2024-03-04T18:26:46.792 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_contentlength_negative PASSED [ 16%] 2024-03-04T18:26:48.223 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws2 PASSED [ 16%] 2024-03-04T18:26:49.907 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_ua_empty_aws2 PASSED [ 17%] 2024-03-04T18:26:51.473 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_ua_none_aws2 PASSED [ 17%] 2024-03-04T18:26:52.867 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_invalid_aws2 PASSED [ 17%] 2024-03-04T18:26:54.314 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_empty_aws2 PASSED [ 17%] 2024-03-04T18:26:55.705 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_before_today_aws2 PASSED [ 17%] 2024-03-04T18:26:57.063 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws2 PASSED [ 17%] 2024-03-04T18:26:58.497 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_after_end_aws2 PASSED [ 17%] 2024-03-04T18:26:59.733 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_ua_empty_aws2 PASSED [ 18%] 2024-03-04T18:27:01.031 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_ua_none_aws2 PASSED [ 18%] 2024-03-04T18:27:01.982 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws2 PASSED [ 18%] 2024-03-04T18:27:02.877 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_empty_aws2 PASSED [ 18%] 2024-03-04T18:27:03.813 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws2 PASSED [ 18%] 2024-03-04T18:27:04.803 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws2 PASSED [ 18%] 2024-03-04T18:27:05.743 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws2 PASSED [ 19%] 2024-03-04T18:27:07.515 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy PASSED [ 19%] 2024-03-04T18:27:08.496 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy_invalid_user PASSED [ 19%] 2024-03-04T18:27:09.562 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy_parameter_limit PASSED [ 19%] 2024-03-04T18:27:10.839 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_put_existing_user_policy PASSED [ 19%] 2024-03-04T18:27:12.042 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_list_user_policy PASSED [ 19%] 2024-03-04T18:27:12.995 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_list_user_policy_invalid_user PASSED [ 19%] 2024-03-04T18:27:14.265 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy PASSED [ 20%] 2024-03-04T18:27:15.470 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy_invalid_user PASSED [ 20%] 2024-03-04T18:27:16.931 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy_from_multiple_policies PASSED [ 20%] 2024-03-04T18:27:18.080 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy PASSED [ 20%] 2024-03-04T18:27:19.313 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_invalid_user PASSED [ 20%] 2024-03-04T18:27:20.562 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_invalid_policy_name PASSED [ 20%] 2024-03-04T18:27:22.355 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_from_multiple_policies PASSED [ 20%] 2024-03-04T18:27:24.308 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_allow_bucket_actions_in_user_policy PASSED [ 21%] 2024-03-04T18:27:26.015 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_deny_bucket_actions_in_user_policy PASSED [ 21%] 2024-03-04T18:27:27.935 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_allow_object_actions_in_user_policy PASSED [ 21%] 2024-03-04T18:27:29.988 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_deny_object_actions_in_user_policy PASSED [ 21%] 2024-03-04T18:27:34.255 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_allow_multipart_actions_in_user_policy PASSED [ 21%] 2024-03-04T18:27:38.494 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_deny_multipart_actions_in_user_policy PASSED [ 21%] 2024-03-04T18:27:40.710 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_allow_tagging_actions_in_user_policy PASSED [ 22%] 2024-03-04T18:27:42.761 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_deny_tagging_actions_in_user_policy PASSED [ 22%] 2024-03-04T18:27:44.410 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_verify_conflicting_user_policy_statements PASSED [ 22%] 2024-03-04T18:27:46.340 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_verify_conflicting_user_policies PASSED [ 22%] 2024-03-04T18:27:47.728 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_iam.py::test_verify_allow_iam_actions PASSED [ 22%] 2024-03-04T18:27:49.623 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_empty PASSED [ 22%] 2024-03-04T18:27:51.571 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_distinct PASSED [ 22%] 2024-03-04T18:27:53.477 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_many PASSED [ 23%] 2024-03-04T18:27:55.387 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_many PASSED [ 23%] 2024-03-04T18:27:57.277 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_basic_key_count PASSED [ 23%] 2024-03-04T18:27:59.244 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_basic PASSED [ 23%] 2024-03-04T18:28:01.130 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_basic PASSED [ 23%] 2024-03-04T18:28:03.037 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_encoding_basic PASSED [ 23%] 2024-03-04T18:28:04.989 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_encoding_basic PASSED [ 23%] 2024-03-04T18:28:08.258 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix PASSED [ 24%] 2024-03-04T18:28:11.614 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix PASSED [ 24%] 2024-03-04T18:28:13.226 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_ends_with_delimiter PASSED [ 24%] 2024-03-04T18:28:14.826 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix_ends_with_delimiter PASSED [ 24%] 2024-03-04T18:28:16.706 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_alt PASSED [ 24%] 2024-03-04T18:28:18.586 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_alt PASSED [ 24%] 2024-03-04T18:28:21.936 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix_underscore PASSED [ 25%] 2024-03-04T18:28:25.241 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_underscore PASSED [ 25%] 2024-03-04T18:28:27.146 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_percentage PASSED [ 25%] 2024-03-04T18:28:29.043 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_percentage PASSED [ 25%] 2024-03-04T18:28:30.950 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_whitespace PASSED [ 25%] 2024-03-04T18:28:32.845 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_whitespace PASSED [ 25%] 2024-03-04T18:28:34.832 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_dot PASSED [ 25%] 2024-03-04T18:28:36.787 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_dot PASSED [ 26%] 2024-03-04T18:28:38.653 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_unreadable PASSED [ 26%] 2024-03-04T18:28:40.643 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_unreadable PASSED [ 26%] 2024-03-04T18:28:42.596 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_empty PASSED [ 26%] 2024-03-04T18:28:44.559 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_empty PASSED [ 26%] 2024-03-04T18:28:46.469 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_none PASSED [ 26%] 2024-03-04T18:28:48.467 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_none PASSED [ 26%] 2024-03-04T18:28:50.311 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_notempty PASSED [ 27%] 2024-03-04T18:28:52.152 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_defaultempty PASSED [ 27%] 2024-03-04T18:28:53.937 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_empty PASSED [ 27%] 2024-03-04T18:28:55.914 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_not_exist PASSED [ 27%] 2024-03-04T18:28:57.785 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_not_exist PASSED [ 27%] 2024-03-04T18:30:47.127 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_not_skip_special PASSED [ 27%] 2024-03-04T18:30:48.939 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_basic PASSED [ 28%] 2024-03-04T18:30:50.720 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_basic PASSED [ 28%] 2024-03-04T18:30:52.530 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_alt PASSED [ 28%] 2024-03-04T18:30:54.316 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_alt PASSED [ 28%] 2024-03-04T18:30:56.150 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_empty PASSED [ 28%] 2024-03-04T18:30:57.961 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_empty PASSED [ 28%] 2024-03-04T18:30:59.786 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_none PASSED [ 28%] 2024-03-04T18:31:01.640 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_none PASSED [ 29%] 2024-03-04T18:31:03.454 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_not_exist PASSED [ 29%] 2024-03-04T18:31:05.268 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_not_exist PASSED [ 29%] 2024-03-04T18:31:07.058 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_unreadable PASSED [ 29%] 2024-03-04T18:31:08.854 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_unreadable PASSED [ 29%] 2024-03-04T18:31:10.838 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_basic PASSED [ 29%] 2024-03-04T18:31:12.738 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_basic PASSED [ 30%] 2024-03-04T18:31:14.629 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_alt PASSED [ 30%] 2024-03-04T18:31:16.606 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_alt PASSED [ 30%] 2024-03-04T18:31:18.451 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_not_exist PASSED [ 30%] 2024-03-04T18:31:20.373 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_not_exist PASSED [ 30%] 2024-03-04T18:31:22.314 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_delimiter_not_exist PASSED [ 30%] 2024-03-04T18:31:24.180 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_delimiter_not_exist PASSED [ 30%] 2024-03-04T18:31:26.326 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 31%] 2024-03-04T18:31:28.465 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 31%] 2024-03-04T18:31:30.719 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_one PASSED [ 31%] 2024-03-04T18:31:32.963 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_one PASSED [ 31%] 2024-03-04T18:31:34.869 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_zero PASSED [ 31%] 2024-03-04T18:31:36.786 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_zero PASSED [ 31%] 2024-03-04T18:31:38.709 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_none PASSED [ 31%] 2024-03-04T18:31:40.617 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_none PASSED [ 32%] 2024-03-04T18:31:41.758 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_account_usage PASSED [ 32%] 2024-03-04T18:31:43.350 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_head_bucket_usage PASSED [ 32%] 2024-03-04T18:31:47.982 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_unordered PASSED [ 32%] 2024-03-04T18:31:52.667 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_unordered PASSED [ 32%] 2024-03-04T18:31:54.560 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_invalid PASSED [ 32%] 2024-03-04T18:31:56.546 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_none PASSED [ 33%] 2024-03-04T18:31:58.427 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_empty PASSED [ 33%] 2024-03-04T18:32:00.388 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_continuationtoken_empty PASSED [ 33%] 2024-03-04T18:32:02.413 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_continuationtoken PASSED [ 33%] 2024-03-04T18:32:04.443 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_both_continuationtoken_startafter PASSED [ 33%] 2024-03-04T18:32:06.427 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_unreadable PASSED [ 33%] 2024-03-04T18:32:08.321 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_unreadable PASSED [ 33%] 2024-03-04T18:32:10.263 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_not_in_list PASSED [ 34%] 2024-03-04T18:32:12.145 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_not_in_list PASSED [ 34%] 2024-03-04T18:32:13.972 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_after_list PASSED [ 34%] 2024-03-04T18:32:15.818 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_after_list PASSED [ 34%] 2024-03-04T18:32:18.202 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data PASSED [ 34%] 2024-03-04T18:32:21.286 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data_versioning PASSED [ 34%] 2024-03-04T18:32:22.866 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_objects_anonymous PASSED [ 34%] 2024-03-04T18:32:24.469 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_objects_anonymous PASSED [ 35%] 2024-03-04T18:32:25.926 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_objects_anonymous_fail PASSED [ 35%] 2024-03-04T18:32:27.374 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_objects_anonymous_fail PASSED [ 35%] 2024-03-04T18:32:28.410 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_notexist PASSED [ 35%] 2024-03-04T18:32:29.462 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_notexist PASSED [ 35%] 2024-03-04T18:32:30.374 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_delete_notexist PASSED [ 35%] 2024-03-04T18:32:31.938 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_delete_nonempty PASSED [ 36%] 2024-03-04T18:32:40.839 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_concurrent_set_canned_acl PASSED [ 36%] 2024-03-04T18:32:41.759 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket PASSED [ 36%] 2024-03-04T18:32:43.349 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding PASSED [ 36%] 2024-03-04T18:32:44.712 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_delete PASSED [ 36%] 2024-03-04T18:32:46.039 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_read_not_exist PASSED [ 36%] 2024-03-04T18:32:47.453 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error PASSED [ 36%] 2024-03-04T18:32:53.031 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete PASSED [ 37%] 2024-03-04T18:32:55.156 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete PASSED [ 37%] 2024-03-04T18:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete PASSED [ 37%] 2024-03-04T18:34:48.114 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit PASSED [ 37%] 2024-03-04T18:36:38.840 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit PASSED [ 37%] 2024-03-04T18:36:40.433 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes PASSED [ 37%] 2024-03-04T18:36:41.947 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_check_etag PASSED [ 38%] 2024-03-04T18:36:43.576 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_cache_control PASSED [ 38%] 2024-03-04T18:36:45.138 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_expires PASSED [ 38%] 2024-03-04T18:36:46.956 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete PASSED [ 38%] 2024-03-04T18:36:48.475 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good PASSED [ 38%] 2024-03-04T18:36:50.109 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty PASSED [ 38%] 2024-03-04T18:36:51.982 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty PASSED [ 38%] 2024-03-04T18:36:53.682 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put PASSED [ 39%] 2024-03-04T18:36:55.385 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_file PASSED [ 39%] 2024-03-04T18:36:57.112 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request PASSED [ 39%] 2024-03-04T18:36:58.971 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request PASSED [ 39%] 2024-03-04T18:37:00.612 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type PASSED [ 39%] 2024-03-04T18:37:01.983 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key PASSED [ 39%] 2024-03-04T18:37:03.521 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code PASSED [ 39%] 2024-03-04T18:37:05.062 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code PASSED [ 40%] 2024-03-04T18:37:08.557 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk PASSED [ 40%] 2024-03-04T18:37:10.443 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename PASSED [ 40%] 2024-03-04T18:37:11.935 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header PASSED [ 40%] 2024-03-04T18:37:13.487 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields PASSED [ 40%] 2024-03-04T18:37:15.226 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values PASSED [ 40%] 2024-03-04T18:37:16.861 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action PASSED [ 41%] 2024-03-04T18:37:18.267 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature PASSED [ 41%] 2024-03-04T18:37:19.676 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key PASSED [ 41%] 2024-03-04T18:37:21.147 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format PASSED [ 41%] 2024-03-04T18:37:22.536 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified PASSED [ 41%] 2024-03-04T18:37:23.989 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature PASSED [ 41%] 2024-03-04T18:37:25.377 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition PASSED [ 41%] 2024-03-04T18:37:27.054 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header PASSED [ 42%] 2024-03-04T18:37:28.433 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field PASSED [ 42%] 2024-03-04T18:37:29.798 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive PASSED [ 42%] 2024-03-04T18:37:31.205 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive PASSED [ 42%] 2024-03-04T18:37:32.592 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy PASSED [ 42%] 2024-03-04T18:37:34.432 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket PASSED [ 42%] 2024-03-04T18:37:35.873 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value PASSED [ 42%] 2024-03-04T18:37:37.348 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition PASSED [ 43%] 2024-03-04T18:37:38.739 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list PASSED [ 43%] 2024-03-04T18:37:40.099 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded PASSED [ 43%] 2024-03-04T18:37:41.523 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument PASSED [ 43%] 2024-03-04T18:37:42.970 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument PASSED [ 43%] 2024-03-04T18:37:44.392 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum PASSED [ 43%] 2024-03-04T18:37:48.079 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug PASSED [ 44%] 2024-03-04T18:37:49.483 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions PASSED [ 44%] 2024-03-04T18:37:51.163 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good PASSED [ 44%] 2024-03-04T18:37:52.845 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed PASSED [ 44%] 2024-03-04T18:37:54.450 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good PASSED [ 44%] 2024-03-04T18:37:56.117 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed PASSED [ 44%] 2024-03-04T18:37:57.753 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good PASSED [ 44%] 2024-03-04T18:38:00.571 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed PASSED [ 45%] 2024-03-04T18:38:02.153 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good PASSED [ 45%] 2024-03-04T18:38:03.731 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed PASSED [ 45%] 2024-03-04T18:38:05.608 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good PASSED [ 45%] 2024-03-04T18:38:07.518 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed PASSED [ 45%] 2024-03-04T18:38:09.324 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good PASSED [ 45%] 2024-03-04T18:38:10.925 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed PASSED [ 46%] 2024-03-04T18:38:12.709 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good PASSED [ 46%] 2024-03-04T18:38:14.559 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed PASSED [ 46%] 2024-03-04T18:38:16.160 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good PASSED [ 46%] 2024-03-04T18:38:17.986 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed PASSED [ 46%] 2024-03-04T18:38:19.588 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get PASSED [ 46%] 2024-03-04T18:38:21.222 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone PASSED [ 46%] 2024-03-04T18:38:22.775 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone PASSED [ 47%] 2024-03-04T18:38:24.467 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone PASSED [ 47%] 2024-03-04T18:38:25.873 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head PASSED [ 47%] 2024-03-04T18:38:26.793 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist PASSED [ 47%] 2024-03-04T18:38:28.864 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_extended PASSED [ 47%] 2024-03-04T18:38:30.550 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl PASSED [ 47%] 2024-03-04T18:38:32.166 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl PASSED [ 47%] 2024-03-04T18:38:36.732 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime PASSED [ 48%] 2024-03-04T18:38:38.357 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated PASSED [ 48%] 2024-03-04T18:38:39.976 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers PASSED [ 48%] 2024-03-04T18:38:41.608 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl PASSED [ 48%] 2024-03-04T18:38:43.102 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl PASSED [ 48%] 2024-03-04T18:38:44.602 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone PASSED [ 48%] 2024-03-04T18:38:46.207 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone PASSED [ 49%] 2024-03-04T18:38:47.956 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired PASSED [ 49%] 2024-03-04T18:38:49.748 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant PASSED [ 49%] 2024-03-04T18:38:51.358 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero PASSED [ 49%] 2024-03-04T18:38:52.935 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range PASSED [ 49%] 2024-03-04T18:38:54.487 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range PASSED [ 49%] 2024-03-04T18:38:56.220 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put PASSED [ 49%] 2024-03-04T18:38:57.922 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access PASSED [ 50%] 2024-03-04T18:38:59.430 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_authenticated PASSED [ 50%] 2024-03-04T18:39:01.041 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired PASSED [ 50%] 2024-03-04T18:39:02.094 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha PASSED [ 50%] 2024-03-04T18:39:03.044 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one PASSED [ 50%] 2024-03-04T18:39:03.996 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two PASSED [ 50%] 2024-03-04T18:39:05.247 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 PASSED [ 50%] 2024-03-04T18:39:06.532 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 PASSED [ 51%] 2024-03-04T18:39:07.738 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 PASSED [ 51%] 2024-03-04T18:39:08.920 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 PASSED [ 51%] 2024-03-04T18:39:10.267 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name PASSED [ 51%] 2024-03-04T18:39:11.225 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip PASSED [ 51%] 2024-03-04T18:39:12.159 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore PASSED [ 51%] 2024-03-04T18:39:13.410 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long PASSED [ 52%] 2024-03-04T18:39:14.355 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end PASSED [ 52%] 2024-03-04T18:39:15.251 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot PASSED [ 52%] 2024-03-04T18:39:16.178 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash PASSED [ 52%] 2024-03-04T18:39:17.157 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot PASSED [ 52%] 2024-03-04T18:39:18.452 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists PASSED [ 52%] 2024-03-04T18:39:19.762 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_get_location PASSED [ 52%] 2024-03-04T18:39:21.174 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner PASSED [ 53%] 2024-03-04T18:39:22.528 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl PASSED [ 53%] 2024-03-04T18:39:23.891 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl PASSED [ 53%] 2024-03-04T18:39:25.236 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_default PASSED [ 53%] 2024-03-04T18:39:26.578 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create PASSED [ 53%] 2024-03-04T18:39:28.032 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned PASSED [ 53%] 2024-03-04T18:39:29.372 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite PASSED [ 53%] 2024-03-04T18:39:30.641 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread PASSED [ 54%] 2024-03-04T18:39:32.294 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_default PASSED [ 54%] 2024-03-04T18:39:33.912 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create PASSED [ 54%] 2024-03-04T18:39:35.710 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned PASSED [ 54%] 2024-03-04T18:39:37.371 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite PASSED [ 54%] 2024-03-04T18:39:39.054 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread PASSED [ 54%] 2024-03-04T18:39:40.861 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread PASSED [ 55%] 2024-03-04T18:39:42.697 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol PASSED [ 55%] 2024-03-04T18:39:44.518 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner PASSED [ 55%] 2024-03-04T18:39:46.335 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes PASSED [ 55%] 2024-03-04T18:39:47.743 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private PASSED [ 55%] 2024-03-04T18:39:49.584 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl PASSED [ 55%] 2024-03-04T18:39:51.369 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_write PASSED [ 55%] 2024-03-04T18:39:53.301 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp PASSED [ 56%] 2024-03-04T18:39:55.116 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_read PASSED [ 56%] 2024-03-04T18:39:56.992 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_readacp PASSED [ 56%] 2024-03-04T18:39:59.610 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol PASSED [ 56%] 2024-03-04T18:40:02.008 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read PASSED [ 56%] 2024-03-04T18:40:04.404 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp PASSED [ 56%] 2024-03-04T18:40:06.937 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write PASSED [ 57%] 2024-03-04T18:40:09.281 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp PASSED [ 57%] 2024-03-04T18:40:10.858 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user PASSED [ 57%] 2024-03-04T18:40:13.141 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_no_grants PASSED [ 57%] 2024-03-04T18:40:14.792 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants PASSED [ 57%] 2024-03-04T18:40:16.493 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants PASSED [ 57%] 2024-03-04T18:40:18.236 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email PASSED [ 57%] 2024-03-04T18:40:19.855 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist PASSED [ 58%] 2024-03-04T18:40:21.755 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all PASSED [ 58%] 2024-03-04T18:40:24.381 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private PASSED [ 58%] 2024-03-04T18:40:26.926 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private PASSED [ 58%] 2024-03-04T18:40:29.591 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread PASSED [ 58%] 2024-03-04T18:40:32.173 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread PASSED [ 58%] 2024-03-04T18:40:34.879 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite PASSED [ 58%] 2024-03-04T18:40:37.360 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite PASSED [ 59%] 2024-03-04T18:40:40.037 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private PASSED [ 59%] 2024-03-04T18:40:42.696 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread PASSED [ 59%] 2024-03-04T18:40:45.265 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite PASSED [ 59%] 2024-03-04T18:40:47.726 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private PASSED [ 59%] 2024-03-04T18:40:50.247 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread PASSED [ 59%] 2024-03-04T18:40:52.736 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite PASSED [ 60%] 2024-03-04T18:40:55.636 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list PASSED [ 60%] 2024-03-04T18:40:58.385 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime PASSED [ 60%] 2024-03-04T18:40:59.338 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous PASSED [ 60%] 2024-03-04T18:41:00.240 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth PASSED [ 60%] 2024-03-04T18:41:01.176 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth PASSED [ 60%] 2024-03-04T18:41:02.848 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha PASSED [ 60%] 2024-03-04T18:41:04.425 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit PASSED [ 61%] 2024-03-04T18:41:05.611 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period PASSED [ 61%] 2024-03-04T18:41:06.867 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen PASSED [ 61%] 2024-03-04T18:41:08.983 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding PASSED [ 61%] 2024-03-04T18:41:13.981 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names PASSED [ 61%] 2024-03-04T18:41:16.216 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix PASSED [ 61%] 2024-03-04T18:41:18.188 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size PASSED [ 61%] 2024-03-04T18:41:27.488 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_16m PASSED [ 62%] 2024-03-04T18:41:29.555 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket PASSED [ 62%] 2024-03-04T18:41:31.857 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype PASSED [ 62%] 2024-03-04T18:41:33.598 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself PASSED [ 62%] 2024-03-04T18:41:35.386 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata PASSED [ 62%] 2024-03-04T18:41:37.822 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket PASSED [ 62%] 2024-03-04T18:41:39.769 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket PASSED [ 63%] 2024-03-04T18:41:42.348 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket PASSED [ 63%] 2024-03-04T18:41:44.508 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl PASSED [ 63%] 2024-03-04T18:41:47.979 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata PASSED [ 63%] 2024-03-04T18:41:51.315 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata PASSED [ 63%] 2024-03-04T18:41:52.714 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found PASSED [ 63%] 2024-03-04T18:41:54.058 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found PASSED [ 63%] 2024-03-04T18:41:59.095 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket PASSED [ 64%] 2024-03-04T18:42:01.320 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding PASSED [ 64%] 2024-03-04T18:42:43.031 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload PASSED [ 64%] 2024-03-04T18:42:44.606 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty PASSED [ 64%] 2024-03-04T18:42:46.840 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_small PASSED [ 64%] 2024-03-04T18:42:54.164 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_small PASSED [ 64%] 2024-03-04T18:42:56.173 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range PASSED [ 65%] 2024-03-04T18:42:59.403 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range PASSED [ 65%] 2024-03-04T18:43:21.952 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names PASSED [ 65%] 2024-03-04T18:44:02.291 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload PASSED [ 65%] 2024-03-04T18:44:50.171 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned PASSED [ 65%] 2024-03-04T18:48:13.212 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part PASSED [ 65%] 2024-03-04T18:48:40.575 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes PASSED [ 65%] 2024-03-04T18:49:17.214 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes PASSED [ 66%] 2024-03-04T18:49:20.052 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small PASSED [ 66%] 2024-03-04T18:49:34.658 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents PASSED [ 66%] 2024-03-04T18:49:45.863 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object PASSED [ 66%] 2024-03-04T18:49:52.639 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload PASSED [ 66%] 2024-03-04T18:49:54.320 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found PASSED [ 66%] 2024-03-04T18:50:05.154 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload PASSED [ 66%] 2024-03-04T18:50:07.270 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner PASSED [ 67%] 2024-03-04T18:50:09.001 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part PASSED [ 67%] 2024-03-04T18:50:10.735 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag PASSED [ 67%] 2024-03-04T18:50:39.152 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_get_part PASSED [ 67%] 2024-03-04T18:50:40.936 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part PASSED [ 67%] 2024-03-04T18:50:42.571 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_100_continue PASSED [ 67%] 2024-03-04T18:50:44.441 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_set_cors PASSED [ 68%] 2024-03-04T18:50:54.142 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_response PASSED [ 68%] 2024-03-04T18:50:58.998 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_wildcard PASSED [ 68%] 2024-03-04T18:51:03.647 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_cors_header_option PASSED [ 68%] 2024-03-04T18:51:05.482 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object PASSED [ 68%] 2024-03-04T18:51:07.277 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object_tenant PASSED [ 68%] 2024-03-04T18:51:08.969 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_set_bucket_tagging PASSED [ 68%] 2024-03-04T18:51:12.520 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_1mb PASSED [ 69%] 2024-03-04T18:51:20.158 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_4mb PASSED [ 69%] 2024-03-04T18:51:33.942 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_8mb PASSED [ 69%] 2024-03-04T18:51:38.756 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_1mb PASSED [ 69%] 2024-03-04T18:51:48.174 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_4mb PASSED [ 69%] 2024-03-04T18:52:03.352 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_8mb PASSED [ 69%] 2024-03-04T18:52:07.438 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_1mb PASSED [ 69%] 2024-03-04T18:52:16.242 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_4mb PASSED [ 70%] 2024-03-04T18:52:31.479 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_8mb PASSED [ 70%] 2024-03-04T18:52:35.500 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_conditional_write_1mb PASSED [ 70%] 2024-03-04T18:52:37.432 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_multipart_upload_write PASSED [ 70%] 2024-03-04T18:52:40.276 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_resend_first_finishes_last PASSED [ 70%] 2024-03-04T18:52:41.901 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_response_code PASSED [ 70%] 2024-03-04T18:52:52.082 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_big_request_response_code PASSED [ 71%] 2024-03-04T18:52:53.751 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code PASSED [ 71%] 2024-03-04T18:52:55.385 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code PASSED [ 71%] 2024-03-04T18:52:57.036 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_invalid_range PASSED [ 71%] 2024-03-04T18:52:58.683 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_empty_object PASSED [ 71%] 2024-03-04T18:53:01.253 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_create_suspend PASSED [ 71%] 2024-03-04T18:53:22.193 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove PASSED [ 71%] 2024-03-04T18:53:25.233 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove_head PASSED [ 72%] 2024-03-04T18:53:27.583 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_removal PASSED [ 72%] 2024-03-04T18:53:30.289 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite PASSED [ 72%] 2024-03-04T18:53:33.073 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended PASSED [ 72%] 2024-03-04T18:53:42.652 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspend_versions PASSED [ 72%] 2024-03-04T18:53:53.118 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_all PASSED [ 72%] 2024-03-04T18:54:30.468 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names PASSED [ 73%] 2024-03-04T18:55:29.459 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart PASSED [ 73%] 2024-03-04T18:55:34.642 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_list_marker PASSED [ 73%] 2024-03-04T18:55:39.601 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_copy_obj_version PASSED [ 73%] 2024-03-04T18:55:42.026 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete PASSED [ 73%] 2024-03-04T18:55:44.628 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker PASSED [ 73%] 2024-03-04T18:55:46.592 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create PASSED [ 73%] 2024-03-04T18:55:49.663 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl PASSED [ 74%] 2024-03-04T18:55:52.517 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl_no_version_specified PASSED [ 74%] 2024-03-04T18:55:57.963 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove PASSED [ 74%] 2024-03-04T18:55:59.591 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set PASSED [ 74%] 2024-03-04T18:56:01.221 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get PASSED [ 74%] 2024-03-04T18:56:02.793 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get_no_id PASSED [ 74%] 2024-03-04T18:56:04.228 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_id_too_long PASSED [ 74%] 2024-03-04T18:56:05.640 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_same_id PASSED [ 75%] 2024-03-04T18:56:07.209 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_invalid_status PASSED [ 75%] 2024-03-04T18:56:08.715 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_date PASSED [ 75%] 2024-03-04T18:56:10.090 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_invalid_date PASSED [ 75%] 2024-03-04T18:56:11.813 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_noncurrent PASSED [ 75%] 2024-03-04T18:56:13.269 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_deletemarker PASSED [ 75%] 2024-03-04T18:56:14.764 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_filter PASSED [ 76%] 2024-03-04T18:56:16.203 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_empty_filter PASSED [ 76%] 2024-03-04T18:56:17.647 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_multipart PASSED [ 76%] 2024-03-04T18:56:19.144 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_transition_set_invalid_date PASSED [ 76%] 2024-03-04T18:56:20.973 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1b PASSED [ 76%] 2024-03-04T18:56:22.606 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1kb PASSED [ 76%] 2024-03-04T18:56:25.176 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1MB PASSED [ 76%] 2024-03-04T18:56:26.911 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_13b PASSED [ 77%] 2024-03-04T18:56:28.662 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_method_head PASSED [ 77%] 2024-03-04T18:56:30.311 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_present PASSED [ 77%] 2024-03-04T18:56:32.050 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_other_key PASSED [ 77%] 2024-03-04T18:56:33.488 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_invalid_md5 PASSED [ 77%] 2024-03-04T18:56:35.095 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_md5 PASSED [ 77%] 2024-03-04T18:56:36.591 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_key PASSED [ 77%] 2024-03-04T18:56:38.059 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_key_no_sse_c PASSED [ 78%] 2024-03-04T18:57:47.799 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_upload PASSED [ 78%] 2024-03-04T18:58:57.463 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload PASSED [ 78%] 2024-03-04T18:59:18.667 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download PASSED [ 78%] 2024-03-04T18:59:20.439 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request PASSED [ 78%] 2024-03-04T18:59:22.218 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_method_head PASSED [ 78%] 2024-03-04T18:59:23.871 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_present PASSED [ 79%] 2024-03-04T18:59:25.556 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_no_key PASSED [ 79%] 2024-03-04T18:59:26.961 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_not_declared PASSED [ 79%] 2024-03-04T19:00:17.605 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_upload PASSED [ 79%] 2024-03-04T19:00:37.026 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 PASSED [ 79%] 2024-03-04T19:00:56.566 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 PASSED [ 79%] 2024-03-04T19:00:58.400 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_post_object_authenticated_request PASSED [ 79%] 2024-03-04T19:01:00.018 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1b PASSED [ 80%] 2024-03-04T19:01:01.642 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1kb PASSED [ 80%] 2024-03-04T19:01:04.256 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1MB PASSED [ 80%] 2024-03-04T19:01:05.928 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_13b PASSED [ 80%] 2024-03-04T19:01:07.610 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_read_declare PASSED [ 80%] 2024-03-04T19:01:09.430 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy PASSED [ 80%] 2024-03-04T19:01:11.206 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy PASSED [ 80%] 2024-03-04T19:01:13.268 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_acl PASSED [ 81%] 2024-03-04T19:01:15.285 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_acl PASSED [ 81%] 2024-03-04T19:01:18.202 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_another_bucket PASSED [ 81%] 2024-03-04T19:01:21.018 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_another_bucket PASSED [ 81%] 2024-03-04T19:01:26.209 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_tagging PASSED [ 81%] 2024-03-04T19:01:32.682 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_head_tagging PASSED [ 81%] 2024-03-04T19:01:38.815 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_tags PASSED [ 82%] 2024-03-04T19:01:44.011 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_tags PASSED [ 82%] 2024-03-04T19:01:49.294 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_kvsize_tags PASSED [ 82%] 2024-03-04T19:01:54.922 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_key_tags PASSED [ 82%] 2024-03-04T19:02:00.594 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_val_tags PASSED [ 82%] 2024-03-04T19:02:05.884 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_modify_tags PASSED [ 82%] 2024-03-04T19:02:11.311 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_delete_tags PASSED [ 82%] 2024-03-04T19:02:13.003 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_anonymous_request PASSED [ 83%] 2024-03-04T19:02:14.803 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_authenticated_request PASSED [ 83%] 2024-03-04T19:02:16.498 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_with_tags PASSED [ 83%] 2024-03-04T19:02:22.385 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_tags_acl_public PASSED [ 83%] 2024-03-04T19:02:27.938 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_tags_acl_public PASSED [ 83%] 2024-03-04T19:02:34.006 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_delete_tags_obj_public PASSED [ 83%] 2024-03-04T19:02:37.795 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id PASSED [ 84%] 2024-03-04T19:03:29.113 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id PASSED [ 84%] 2024-03-04T19:03:31.790 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag PASSED [ 84%] 2024-03-04T19:03:34.519 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag PASSED [ 84%] 2024-03-04T19:03:37.116 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag PASSED [ 84%] 2024-03-04T19:03:40.409 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source PASSED [ 84%] 2024-03-04T19:03:43.230 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta PASSED [ 84%] 2024-03-04T19:03:45.161 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_acl PASSED [ 85%] 2024-03-04T19:03:47.851 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_grant PASSED [ 85%] 2024-03-04T19:03:49.268 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 PASSED [ 85%] 2024-03-04T19:03:50.728 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_kms PASSED [ 85%] 2024-03-04T19:03:52.096 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms PASSED [ 85%] 2024-03-04T19:03:53.472 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms PASSED [ 85%] 2024-03-04T19:03:55.261 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc PASSED [ 85%] 2024-03-04T19:03:56.764 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 PASSED [ 86%] 2024-03-04T19:03:59.403 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag PASSED [ 86%] 2024-03-04T19:04:00.925 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock PASSED [ 86%] 2024-03-04T19:04:02.259 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket PASSED [ 86%] 2024-03-04T19:04:03.601 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years PASSED [ 86%] 2024-03-04T19:04:05.055 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days PASSED [ 86%] 2024-03-04T19:04:06.407 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years PASSED [ 87%] 2024-03-04T19:04:07.819 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode PASSED [ 87%] 2024-03-04T19:04:09.179 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status PASSED [ 87%] 2024-03-04T19:04:10.535 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_suspend_versioning PASSED [ 87%] 2024-03-04T19:04:11.867 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock PASSED [ 87%] 2024-03-04T19:04:13.181 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket PASSED [ 87%] 2024-03-04T19:04:14.852 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention PASSED [ 87%] 2024-03-04T19:04:16.428 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket PASSED [ 88%] 2024-03-04T19:04:18.138 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode PASSED [ 88%] 2024-03-04T19:04:19.838 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention PASSED [ 88%] 2024-03-04T19:04:21.539 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 PASSED [ 88%] 2024-03-04T19:04:23.067 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket PASSED [ 88%] 2024-03-04T19:04:25.000 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_versionid PASSED [ 88%] 2024-03-04T19:04:26.841 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention PASSED [ 88%] 2024-03-04T19:04:28.648 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period PASSED [ 89%] 2024-03-04T19:04:30.458 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period PASSED [ 89%] 2024-03-04T19:04:32.233 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass PASSED [ 89%] 2024-03-04T19:04:33.921 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention PASSED [ 89%] 2024-03-04T19:04:35.965 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker PASSED [ 89%] 2024-03-04T19:04:37.877 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention PASSED [ 89%] 2024-03-04T19:04:39.612 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold PASSED [ 90%] 2024-03-04T19:04:41.274 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket PASSED [ 90%] 2024-03-04T19:04:42.817 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status PASSED [ 90%] 2024-03-04T19:04:44.681 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold PASSED [ 90%] 2024-03-04T19:04:46.340 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket PASSED [ 90%] 2024-03-04T19:04:48.136 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on PASSED [ 90%] 2024-03-04T19:04:49.705 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off PASSED [ 90%] 2024-03-04T19:04:51.615 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_metadata PASSED [ 91%] 2024-03-04T19:04:53.307 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_uploading_obj PASSED [ 91%] 2024-03-04T19:05:04.490 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass PASSED [ 91%] 2024-03-04T19:05:06.277 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass PASSED [ 91%] 2024-03-04T19:05:16.916 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_compliance PASSED [ 91%] 2024-03-04T19:05:18.687 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifmatch_good PASSED [ 91%] 2024-03-04T19:05:20.430 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifnonematch_failed PASSED [ 92%] 2024-03-04T19:05:21.859 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_policy_status PASSED [ 92%] 2024-03-04T19:05:23.371 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_acl_bucket_policy_status PASSED [ 92%] 2024-03-04T19:05:24.842 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status PASSED [ 92%] 2024-03-04T19:05:26.481 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status PASSED [ 92%] 2024-03-04T19:05:28.079 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status PASSED [ 92%] 2024-03-04T19:05:29.700 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_deny_bucket_policy_status PASSED [ 92%] 2024-03-04T19:05:31.129 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_default_public_block PASSED [ 93%] 2024-03-04T19:05:32.627 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_public_block PASSED [ 93%] 2024-03-04T19:05:34.237 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_put_bucket_acls PASSED [ 93%] 2024-03-04T19:05:35.945 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_object_canned_acls PASSED [ 93%] 2024-03-04T19:05:37.627 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_policy PASSED [ 93%] 2024-03-04T19:05:39.706 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_ignore_public_acls PASSED [ 93%] 2024-03-04T19:06:10.648 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy PASSED [ 93%] 2024-03-04T19:06:12.174 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_put_bucket_encryption_kms PASSED [ 94%] 2024-03-04T19:06:13.735 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_encryption_kms PASSED [ 94%] 2024-03-04T19:06:15.359 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_delete_bucket_encryption_kms PASSED [ 94%] 2024-03-04T19:06:17.363 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request PASSED [ 94%] 2024-03-04T19:06:22.335 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_torrent PASSED [ 94%] 2024-03-04T19:06:45.892 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3select.py::test_generate_where_clause PASSED [ 94%] 2024-03-04T19:07:06.998 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3select.py::test_generate_projection PASSED [ 95%] 2024-03-04T19:07:08.625 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3select.py::test_count_operation PASSED [ 95%] 2024-03-04T19:07:11.011 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3select.py::test_count_json_operation PASSED [ 95%] 2024-03-04T19:07:27.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T19:07:27.378 INFO:teuthology.orchestra.run.smithi137.stdout:s3tests_boto3/functional/test_s3select.py::test_json_column_sum_min_max 2024-03-04T19:07:27.378 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 437, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 380, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi137 was lost: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-03-04T19:07:27.380 INFO:tasks.s3tests:Cleaning up boto... 2024-03-04T19:07:27.380 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-04T19:07:27.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T19:07:27.870 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-03-04T19:07:28.319 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-04T19:07:28.319 DEBUG:teuthology.orchestra.run.smithi137:> rm /home/ubuntu/cephtest/boto-client.0.cfg 2024-03-04T19:07:28.368 INFO:tasks.s3tests:Removing s3-tests... 2024-03-04T19:07:28.368 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/s3-tests-client.0 2024-03-04T19:07:28.548 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 703, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 61, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 437, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 380, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi137 was lost: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-03-04T19:07:28.739 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=04fc6cd4689e4c86b7873b5dd8f182f6 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 703, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 61, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 437, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/github.com_pritha-srivastava_ceph_3aec390336fdfdc8f927818b718cfb0361bd47c8/qa/tasks/s3tests.py", line 380, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi137 was lost: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-03-04T19:07:28.742 DEBUG:teuthology.run_tasks:Unwinding manager s3tests 2024-03-04T19:07:28.752 DEBUG:teuthology.run_tasks:Unwinding manager s3tests-java 2024-03-04T19:07:28.764 DEBUG:tasks.s3tests_java:S3 Tests Java: END 2024-03-04T19:07:28.764 INFO:tasks.s3tests_java:S3 Tests Java: Cleaning up s3-tests-java... 2024-03-04T19:07:28.764 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/s3-tests-java 2024-03-04T19:07:28.816 INFO:tasks.s3tests_java:S3 Tests Java: Deleting S3 users... 2024-03-04T19:07:28.817 DEBUG:teuthology.orchestra.run.smithi137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid tester.client.0 --purge-data --cluster ceph 2024-03-04T19:07:28.885 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setuser ceph since I am not root 2024-03-04T19:07:28.886 INFO:teuthology.orchestra.run.smithi137.stderr:ignoring --setgroup ceph since I am not root 2024-03-04T19:07:31.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-03-04T19:07:31.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-03-04T19:07:31.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-03-04T19:07:31.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-03-04T19:07:31.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-03-04T19:07:31.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-03-04T19:07:31.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-03-04T19:07:31.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-03-04T19:07:31.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-03-04T19:07:31.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-03-04T19:07:31.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-03-04T19:07:31.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s 2024-03-04T19:07:31.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-03-04T19:07:31.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-03-04T19:07:36.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~5s 2024-03-04T19:07:36.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~5s 2024-03-04T19:07:36.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~5s 2024-03-04T19:07:36.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~5s 2024-03-04T19:07:36.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~5s 2024-03-04T19:07:36.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~5s 2024-03-04T19:07:36.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~5s 2024-03-04T19:07:36.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~5s 2024-03-04T19:07:36.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~5s 2024-03-04T19:07:36.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~5s 2024-03-04T19:07:36.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~5s 2024-03-04T19:07:36.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~5s 2024-03-04T19:07:36.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~5s 2024-03-04T19:07:36.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~5s 2024-03-04T19:07:41.320 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~10s 2024-03-04T19:07:41.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~10s 2024-03-04T19:07:41.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~10s 2024-03-04T19:07:41.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~10s 2024-03-04T19:07:41.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~10s 2024-03-04T19:07:41.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~10s 2024-03-04T19:07:41.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~10s 2024-03-04T19:07:41.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~10s 2024-03-04T19:07:41.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~10s 2024-03-04T19:07:41.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~10s 2024-03-04T19:07:41.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~10s 2024-03-04T19:07:41.322 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~10s 2024-03-04T19:07:41.322 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~10s 2024-03-04T19:07:41.322 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~10s 2024-03-04T19:07:46.323 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~15s 2024-03-04T19:07:46.323 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~15s 2024-03-04T19:07:46.323 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~15s 2024-03-04T19:07:46.323 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~15s 2024-03-04T19:07:46.323 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~15s 2024-03-04T19:07:46.324 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~15s 2024-03-04T19:07:46.324 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~15s 2024-03-04T19:07:46.324 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~15s 2024-03-04T19:07:46.324 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~15s 2024-03-04T19:07:46.324 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~15s 2024-03-04T19:07:46.324 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~15s 2024-03-04T19:07:46.324 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~15s 2024-03-04T19:07:46.324 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~15s 2024-03-04T19:07:46.324 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~15s 2024-03-04T19:07:51.326 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~20s 2024-03-04T19:07:51.326 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~20s 2024-03-04T19:07:51.327 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~20s 2024-03-04T19:07:51.327 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~20s 2024-03-04T19:07:51.327 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~20s 2024-03-04T19:07:51.327 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~20s 2024-03-04T19:07:51.327 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~20s 2024-03-04T19:07:51.327 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~20s 2024-03-04T19:07:51.327 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~20s 2024-03-04T19:07:51.327 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~20s 2024-03-04T19:07:51.327 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~20s 2024-03-04T19:07:51.327 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~20s 2024-03-04T19:07:51.328 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~20s 2024-03-04T19:07:51.328 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~20s 2024-03-04T19:07:56.328 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~25s 2024-03-04T19:07:56.329 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~25s 2024-03-04T19:07:56.329 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~25s 2024-03-04T19:07:56.329 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~25s 2024-03-04T19:07:56.329 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~25s 2024-03-04T19:07:56.329 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~25s 2024-03-04T19:07:56.329 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~25s 2024-03-04T19:07:56.329 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~25s 2024-03-04T19:07:56.330 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~25s 2024-03-04T19:07:56.330 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~25s 2024-03-04T19:07:56.330 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~25s 2024-03-04T19:07:56.330 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~25s 2024-03-04T19:07:56.330 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~25s 2024-03-04T19:07:56.330 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~25s 2024-03-04T19:08:01.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~30s 2024-03-04T19:08:01.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~30s 2024-03-04T19:08:01.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~30s 2024-03-04T19:08:01.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~30s 2024-03-04T19:08:01.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~30s 2024-03-04T19:08:01.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~30s 2024-03-04T19:08:01.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~30s 2024-03-04T19:08:01.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~30s 2024-03-04T19:08:01.333 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~30s 2024-03-04T19:08:01.333 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~30s 2024-03-04T19:08:01.333 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~30s 2024-03-04T19:08:01.333 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~30s 2024-03-04T19:08:01.333 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~30s 2024-03-04T19:08:01.333 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~30s 2024-03-04T19:08:06.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~35s 2024-03-04T19:08:06.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~35s 2024-03-04T19:08:06.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~35s 2024-03-04T19:08:06.335 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~35s 2024-03-04T19:08:06.335 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~35s 2024-03-04T19:08:06.335 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~35s 2024-03-04T19:08:06.335 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~35s 2024-03-04T19:08:06.335 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~35s 2024-03-04T19:08:06.335 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~35s 2024-03-04T19:08:06.335 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~35s 2024-03-04T19:08:06.335 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~35s 2024-03-04T19:08:06.335 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~35s 2024-03-04T19:08:06.335 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~35s 2024-03-04T19:08:06.336 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~35s 2024-03-04T19:08:11.337 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~40s 2024-03-04T19:08:11.338 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~40s 2024-03-04T19:08:11.338 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~40s 2024-03-04T19:08:11.338 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~40s 2024-03-04T19:08:11.338 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~40s 2024-03-04T19:08:11.338 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~40s 2024-03-04T19:08:11.338 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~40s 2024-03-04T19:08:11.338 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~40s 2024-03-04T19:08:11.338 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~40s 2024-03-04T19:08:11.338 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~40s 2024-03-04T19:08:11.338 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~40s 2024-03-04T19:08:11.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~40s 2024-03-04T19:08:11.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~40s 2024-03-04T19:08:11.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~40s 2024-03-04T19:08:16.340 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~45s 2024-03-04T19:08:16.340 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~45s 2024-03-04T19:08:16.340 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~45s 2024-03-04T19:08:16.340 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~45s 2024-03-04T19:08:16.340 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~45s 2024-03-04T19:08:16.341 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~45s 2024-03-04T19:08:16.341 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~45s 2024-03-04T19:08:16.341 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~45s 2024-03-04T19:08:16.341 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~45s 2024-03-04T19:08:16.341 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~45s 2024-03-04T19:08:16.341 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~45s 2024-03-04T19:08:16.341 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~45s 2024-03-04T19:08:16.341 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~45s 2024-03-04T19:08:16.341 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~45s 2024-03-04T19:08:21.343 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~50s 2024-03-04T19:08:21.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~50s 2024-03-04T19:08:21.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~50s 2024-03-04T19:08:21.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~50s 2024-03-04T19:08:21.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~50s 2024-03-04T19:08:21.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~50s 2024-03-04T19:08:21.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~50s 2024-03-04T19:08:21.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~50s 2024-03-04T19:08:21.345 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~50s 2024-03-04T19:08:21.345 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~50s 2024-03-04T19:08:21.345 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~50s 2024-03-04T19:08:21.345 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~50s 2024-03-04T19:08:21.345 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~50s 2024-03-04T19:08:21.345 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~50s 2024-03-04T19:08:26.346 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~55s 2024-03-04T19:08:26.346 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~55s 2024-03-04T19:08:26.346 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~55s 2024-03-04T19:08:26.347 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~55s 2024-03-04T19:08:26.347 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~55s 2024-03-04T19:08:26.347 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~55s 2024-03-04T19:08:26.347 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~55s 2024-03-04T19:08:26.347 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~55s 2024-03-04T19:08:26.347 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~55s 2024-03-04T19:08:26.347 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~55s 2024-03-04T19:08:26.347 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~55s 2024-03-04T19:08:26.347 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~55s 2024-03-04T19:08:26.347 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~55s 2024-03-04T19:08:26.348 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~55s 2024-03-04T19:08:31.349 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~60s 2024-03-04T19:08:31.349 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~60s 2024-03-04T19:08:31.349 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~60s 2024-03-04T19:08:31.349 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~60s 2024-03-04T19:08:31.349 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~60s 2024-03-04T19:08:31.349 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~60s 2024-03-04T19:08:31.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~60s 2024-03-04T19:08:31.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~60s 2024-03-04T19:08:31.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~60s 2024-03-04T19:08:31.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~60s 2024-03-04T19:08:31.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~60s 2024-03-04T19:08:31.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~60s 2024-03-04T19:08:31.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~60s 2024-03-04T19:08:31.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~60s 2024-03-04T19:08:36.352 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~65s 2024-03-04T19:08:36.352 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~65s 2024-03-04T19:08:36.352 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~65s 2024-03-04T19:08:36.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~65s 2024-03-04T19:08:36.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~65s 2024-03-04T19:08:36.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~65s 2024-03-04T19:08:36.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~65s 2024-03-04T19:08:36.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~65s 2024-03-04T19:08:36.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~65s 2024-03-04T19:08:36.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~65s 2024-03-04T19:08:36.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~65s 2024-03-04T19:08:36.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~65s 2024-03-04T19:08:36.354 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~65s 2024-03-04T19:08:36.354 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~65s 2024-03-04T19:08:41.355 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~70s 2024-03-04T19:08:41.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~70s 2024-03-04T19:08:41.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~70s 2024-03-04T19:08:41.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~70s 2024-03-04T19:08:41.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~70s 2024-03-04T19:08:41.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~70s 2024-03-04T19:08:41.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~70s 2024-03-04T19:08:41.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~70s 2024-03-04T19:08:41.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~70s 2024-03-04T19:08:41.357 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~70s 2024-03-04T19:08:41.357 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~70s 2024-03-04T19:08:41.357 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~70s 2024-03-04T19:08:41.357 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~70s 2024-03-04T19:08:41.357 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~70s 2024-03-04T19:08:46.359 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~75s 2024-03-04T19:08:46.359 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~75s 2024-03-04T19:08:46.359 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~75s 2024-03-04T19:08:46.359 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~75s 2024-03-04T19:08:46.359 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~75s 2024-03-04T19:08:46.359 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~75s 2024-03-04T19:08:46.359 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~75s 2024-03-04T19:08:46.360 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~75s 2024-03-04T19:08:46.360 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~75s 2024-03-04T19:08:46.360 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~75s 2024-03-04T19:08:46.360 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~75s 2024-03-04T19:08:46.360 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~75s 2024-03-04T19:08:46.360 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~75s 2024-03-04T19:08:46.360 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~75s 2024-03-04T19:08:51.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~80s 2024-03-04T19:08:51.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~80s 2024-03-04T19:08:51.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~80s 2024-03-04T19:08:51.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~80s 2024-03-04T19:08:51.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~80s 2024-03-04T19:08:51.362 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~80s 2024-03-04T19:08:51.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~80s 2024-03-04T19:08:51.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~80s 2024-03-04T19:08:51.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~80s 2024-03-04T19:08:51.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~80s 2024-03-04T19:08:51.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~80s 2024-03-04T19:08:51.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~80s 2024-03-04T19:08:51.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~80s 2024-03-04T19:08:51.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~80s 2024-03-04T19:08:56.365 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~85s 2024-03-04T19:08:56.365 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~85s 2024-03-04T19:08:56.365 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~85s 2024-03-04T19:08:56.365 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~85s 2024-03-04T19:08:56.365 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~85s 2024-03-04T19:08:56.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~85s 2024-03-04T19:08:56.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~85s 2024-03-04T19:08:56.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~85s 2024-03-04T19:08:56.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~85s 2024-03-04T19:08:56.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~85s 2024-03-04T19:08:56.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~85s 2024-03-04T19:08:56.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~85s 2024-03-04T19:08:56.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~85s 2024-03-04T19:08:56.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~85s 2024-03-04T19:09:01.368 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~90s 2024-03-04T19:09:01.368 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~90s 2024-03-04T19:09:01.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~90s 2024-03-04T19:09:01.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~90s 2024-03-04T19:09:01.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~90s 2024-03-04T19:09:01.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~90s 2024-03-04T19:09:01.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~90s 2024-03-04T19:09:01.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~90s 2024-03-04T19:09:01.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~90s 2024-03-04T19:09:01.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~90s 2024-03-04T19:09:01.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~90s 2024-03-04T19:09:01.370 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~90s 2024-03-04T19:09:01.370 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~90s 2024-03-04T19:09:01.370 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~90s 2024-03-04T19:09:06.371 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~95s 2024-03-04T19:09:06.371 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~95s 2024-03-04T19:09:06.371 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~95s 2024-03-04T19:09:06.371 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~95s 2024-03-04T19:09:06.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~95s 2024-03-04T19:09:06.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~95s 2024-03-04T19:09:06.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~95s 2024-03-04T19:09:06.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~95s 2024-03-04T19:09:06.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~95s 2024-03-04T19:09:06.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~95s 2024-03-04T19:09:06.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~95s 2024-03-04T19:09:06.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~95s 2024-03-04T19:09:06.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~95s 2024-03-04T19:09:06.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~95s 2024-03-04T19:09:11.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~100s 2024-03-04T19:09:11.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~100s 2024-03-04T19:09:11.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~100s 2024-03-04T19:09:11.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~100s 2024-03-04T19:09:11.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~100s 2024-03-04T19:09:11.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~100s 2024-03-04T19:09:11.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~100s 2024-03-04T19:09:11.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~100s 2024-03-04T19:09:11.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~100s 2024-03-04T19:09:11.376 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~100s 2024-03-04T19:09:11.376 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~100s 2024-03-04T19:09:11.376 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~100s 2024-03-04T19:09:11.376 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~100s 2024-03-04T19:09:11.376 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~100s 2024-03-04T19:09:16.377 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~105s 2024-03-04T19:09:16.378 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~105s 2024-03-04T19:09:16.378 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~105s 2024-03-04T19:09:16.378 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~105s 2024-03-04T19:09:16.378 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~105s 2024-03-04T19:09:16.378 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~105s 2024-03-04T19:09:16.378 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~105s 2024-03-04T19:09:16.378 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~105s 2024-03-04T19:09:16.378 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~105s 2024-03-04T19:09:16.379 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~105s 2024-03-04T19:09:16.379 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~105s 2024-03-04T19:09:16.379 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~105s 2024-03-04T19:09:16.379 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~105s 2024-03-04T19:09:16.379 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~105s 2024-03-04T19:09:21.381 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~110s 2024-03-04T19:09:21.381 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~110s 2024-03-04T19:09:21.382 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~110s 2024-03-04T19:09:21.382 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~110s 2024-03-04T19:09:21.382 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~110s 2024-03-04T19:09:21.382 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~110s 2024-03-04T19:09:21.382 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~110s 2024-03-04T19:09:21.382 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~110s 2024-03-04T19:09:21.382 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~110s 2024-03-04T19:09:21.382 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~110s 2024-03-04T19:09:21.382 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~110s 2024-03-04T19:09:21.383 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~110s 2024-03-04T19:09:21.383 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~110s 2024-03-04T19:09:21.383 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~110s 2024-03-04T19:09:26.384 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~115s 2024-03-04T19:09:26.384 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~115s 2024-03-04T19:09:26.384 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~115s 2024-03-04T19:09:26.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~115s 2024-03-04T19:09:26.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~115s 2024-03-04T19:09:26.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~115s 2024-03-04T19:09:26.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~115s 2024-03-04T19:09:26.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~115s 2024-03-04T19:09:26.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~115s 2024-03-04T19:09:26.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~115s 2024-03-04T19:09:26.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~115s 2024-03-04T19:09:26.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~115s 2024-03-04T19:09:26.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~115s 2024-03-04T19:09:26.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~115s 2024-03-04T19:09:31.387 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~120s 2024-03-04T19:09:31.387 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~120s 2024-03-04T19:09:31.387 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~120s 2024-03-04T19:09:31.387 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~120s 2024-03-04T19:09:31.387 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~120s 2024-03-04T19:09:31.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~120s 2024-03-04T19:09:31.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~120s 2024-03-04T19:09:31.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~120s 2024-03-04T19:09:31.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~120s 2024-03-04T19:09:31.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~120s 2024-03-04T19:09:31.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~120s 2024-03-04T19:09:31.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~120s 2024-03-04T19:09:31.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~120s 2024-03-04T19:09:31.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~120s 2024-03-04T19:09:36.389 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~125s 2024-03-04T19:09:36.390 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~125s 2024-03-04T19:09:36.390 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~125s 2024-03-04T19:09:36.390 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~125s 2024-03-04T19:09:36.390 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~125s 2024-03-04T19:09:36.390 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~125s 2024-03-04T19:09:36.390 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~125s 2024-03-04T19:09:36.390 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~125s 2024-03-04T19:09:36.390 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~125s 2024-03-04T19:09:36.391 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~125s 2024-03-04T19:09:36.391 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~125s 2024-03-04T19:09:36.391 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~125s 2024-03-04T19:09:36.391 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~125s 2024-03-04T19:09:36.391 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~125s 2024-03-04T19:09:41.392 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~130s 2024-03-04T19:09:41.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~130s 2024-03-04T19:09:41.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~130s 2024-03-04T19:09:41.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~130s 2024-03-04T19:09:41.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~130s 2024-03-04T19:09:41.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~130s 2024-03-04T19:09:41.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~130s 2024-03-04T19:09:41.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~130s 2024-03-04T19:09:41.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~130s 2024-03-04T19:09:41.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~130s 2024-03-04T19:09:41.394 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~130s 2024-03-04T19:09:41.394 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~130s 2024-03-04T19:09:41.394 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~130s 2024-03-04T19:09:41.394 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~130s 2024-03-04T19:09:46.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~135s 2024-03-04T19:09:46.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~135s 2024-03-04T19:09:46.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~135s 2024-03-04T19:09:46.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~135s 2024-03-04T19:09:46.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~135s 2024-03-04T19:09:46.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~135s 2024-03-04T19:09:46.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~135s 2024-03-04T19:09:46.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~135s 2024-03-04T19:09:46.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~135s 2024-03-04T19:09:46.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~135s 2024-03-04T19:09:46.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~135s 2024-03-04T19:09:46.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~135s 2024-03-04T19:09:46.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~135s 2024-03-04T19:09:46.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~135s 2024-03-04T19:09:51.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~140s 2024-03-04T19:09:51.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~140s 2024-03-04T19:09:51.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~140s 2024-03-04T19:09:51.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~140s 2024-03-04T19:09:51.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~140s 2024-03-04T19:09:51.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~140s 2024-03-04T19:09:51.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~140s 2024-03-04T19:09:51.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~140s 2024-03-04T19:09:51.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~140s 2024-03-04T19:09:51.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~140s 2024-03-04T19:09:51.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~140s 2024-03-04T19:09:51.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~140s 2024-03-04T19:09:51.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~140s 2024-03-04T19:09:51.400 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~140s 2024-03-04T19:09:56.402 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~145s 2024-03-04T19:09:56.403 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~145s 2024-03-04T19:09:56.403 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~145s 2024-03-04T19:09:56.404 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~145s 2024-03-04T19:09:56.404 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~145s 2024-03-04T19:09:56.404 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~145s 2024-03-04T19:09:56.404 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~145s 2024-03-04T19:09:56.404 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~145s 2024-03-04T19:09:56.404 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~145s 2024-03-04T19:09:56.404 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~145s 2024-03-04T19:09:56.404 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~145s 2024-03-04T19:09:56.404 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~145s 2024-03-04T19:09:56.405 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~145s 2024-03-04T19:09:56.405 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~145s 2024-03-04T19:10:01.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~150s 2024-03-04T19:10:01.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~150s 2024-03-04T19:10:01.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~150s 2024-03-04T19:10:01.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~150s 2024-03-04T19:10:01.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~150s 2024-03-04T19:10:01.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~150s 2024-03-04T19:10:01.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~150s 2024-03-04T19:10:01.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~150s 2024-03-04T19:10:01.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~150s 2024-03-04T19:10:01.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~150s 2024-03-04T19:10:01.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~150s 2024-03-04T19:10:01.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~150s 2024-03-04T19:10:01.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~150s 2024-03-04T19:10:01.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~150s 2024-03-04T19:10:06.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~155s 2024-03-04T19:10:06.409 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~155s 2024-03-04T19:10:06.409 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~155s 2024-03-04T19:10:06.409 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~155s 2024-03-04T19:10:06.409 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~155s 2024-03-04T19:10:06.409 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~155s 2024-03-04T19:10:06.409 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~155s 2024-03-04T19:10:06.409 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~155s 2024-03-04T19:10:06.409 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~155s 2024-03-04T19:10:06.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~155s 2024-03-04T19:10:06.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~155s 2024-03-04T19:10:06.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~155s 2024-03-04T19:10:06.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~155s 2024-03-04T19:10:06.410 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~155s 2024-03-04T19:10:11.411 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~160s 2024-03-04T19:10:11.412 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~160s 2024-03-04T19:10:11.412 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~160s 2024-03-04T19:10:11.412 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~160s 2024-03-04T19:10:11.412 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~160s 2024-03-04T19:10:11.412 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~160s 2024-03-04T19:10:11.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~160s 2024-03-04T19:10:11.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~160s 2024-03-04T19:10:11.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~160s 2024-03-04T19:10:11.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~160s 2024-03-04T19:10:11.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~160s 2024-03-04T19:10:11.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~160s 2024-03-04T19:10:11.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~160s 2024-03-04T19:10:11.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~160s 2024-03-04T19:10:16.415 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~165s 2024-03-04T19:10:16.415 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~165s 2024-03-04T19:10:16.415 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~165s 2024-03-04T19:10:16.415 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~165s 2024-03-04T19:10:16.415 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~165s 2024-03-04T19:10:16.416 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~165s 2024-03-04T19:10:16.416 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~165s 2024-03-04T19:10:16.416 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~165s 2024-03-04T19:10:16.416 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~165s 2024-03-04T19:10:16.416 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~165s 2024-03-04T19:10:16.416 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~165s 2024-03-04T19:10:16.416 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~165s 2024-03-04T19:10:16.416 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~165s 2024-03-04T19:10:16.417 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~165s 2024-03-04T19:10:21.417 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~170s 2024-03-04T19:10:21.418 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~170s 2024-03-04T19:10:21.418 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~170s 2024-03-04T19:10:21.418 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~170s 2024-03-04T19:10:21.418 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~170s 2024-03-04T19:10:21.418 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~170s 2024-03-04T19:10:21.418 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~170s 2024-03-04T19:10:21.418 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~170s 2024-03-04T19:10:21.418 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~170s 2024-03-04T19:10:21.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~170s 2024-03-04T19:10:21.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~170s 2024-03-04T19:10:21.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~170s 2024-03-04T19:10:21.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~170s 2024-03-04T19:10:21.419 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~170s 2024-03-04T19:10:26.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~175s 2024-03-04T19:10:26.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~175s 2024-03-04T19:10:26.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~175s 2024-03-04T19:10:26.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~175s 2024-03-04T19:10:26.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~175s 2024-03-04T19:10:26.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~175s 2024-03-04T19:10:26.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~175s 2024-03-04T19:10:26.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~175s 2024-03-04T19:10:26.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~175s 2024-03-04T19:10:26.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~175s 2024-03-04T19:10:26.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~175s 2024-03-04T19:10:26.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~175s 2024-03-04T19:10:26.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~175s 2024-03-04T19:10:26.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~175s 2024-03-04T19:10:31.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~180s 2024-03-04T19:10:31.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~180s 2024-03-04T19:10:31.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~180s 2024-03-04T19:10:31.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~180s 2024-03-04T19:10:31.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~180s 2024-03-04T19:10:31.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~180s 2024-03-04T19:10:31.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~180s 2024-03-04T19:10:31.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~180s 2024-03-04T19:10:31.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~180s 2024-03-04T19:10:31.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~180s 2024-03-04T19:10:31.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~180s 2024-03-04T19:10:31.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~180s 2024-03-04T19:10:31.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~180s 2024-03-04T19:10:31.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~180s 2024-03-04T19:10:36.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~185s 2024-03-04T19:10:36.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~185s 2024-03-04T19:10:36.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~185s 2024-03-04T19:10:36.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~185s 2024-03-04T19:10:36.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~185s 2024-03-04T19:10:36.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~185s 2024-03-04T19:10:36.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~185s 2024-03-04T19:10:36.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~185s 2024-03-04T19:10:36.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~185s 2024-03-04T19:10:36.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~185s 2024-03-04T19:10:36.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~185s 2024-03-04T19:10:36.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~185s 2024-03-04T19:10:36.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~185s 2024-03-04T19:10:36.428 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~185s 2024-03-04T19:10:41.429 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~190s 2024-03-04T19:10:41.429 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~190s 2024-03-04T19:10:41.429 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~190s 2024-03-04T19:10:41.429 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~190s 2024-03-04T19:10:41.429 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~190s 2024-03-04T19:10:41.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~190s 2024-03-04T19:10:41.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~190s 2024-03-04T19:10:41.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~190s 2024-03-04T19:10:41.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~190s 2024-03-04T19:10:41.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~190s 2024-03-04T19:10:41.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~190s 2024-03-04T19:10:41.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~190s 2024-03-04T19:10:41.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~190s 2024-03-04T19:10:41.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~190s 2024-03-04T19:10:46.431 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~195s 2024-03-04T19:10:46.431 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~195s 2024-03-04T19:10:46.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~195s 2024-03-04T19:10:46.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~195s 2024-03-04T19:10:46.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~195s 2024-03-04T19:10:46.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~195s 2024-03-04T19:10:46.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~195s 2024-03-04T19:10:46.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~195s 2024-03-04T19:10:46.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~195s 2024-03-04T19:10:46.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~195s 2024-03-04T19:10:46.433 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~195s 2024-03-04T19:10:46.433 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~195s 2024-03-04T19:10:46.433 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~195s 2024-03-04T19:10:46.433 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~195s 2024-03-04T19:10:51.434 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~200s 2024-03-04T19:10:51.434 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~200s 2024-03-04T19:10:51.434 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~200s 2024-03-04T19:10:51.434 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~200s 2024-03-04T19:10:51.434 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~200s 2024-03-04T19:10:51.434 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~200s 2024-03-04T19:10:51.435 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~200s 2024-03-04T19:10:51.435 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~200s 2024-03-04T19:10:51.435 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~200s 2024-03-04T19:10:51.435 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~200s 2024-03-04T19:10:51.435 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~200s 2024-03-04T19:10:51.435 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~200s 2024-03-04T19:10:51.435 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~200s 2024-03-04T19:10:51.435 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~200s 2024-03-04T19:10:56.437 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~205s 2024-03-04T19:10:56.437 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~205s 2024-03-04T19:10:56.437 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~205s 2024-03-04T19:10:56.437 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~205s 2024-03-04T19:10:56.438 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~205s 2024-03-04T19:10:56.438 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~205s 2024-03-04T19:10:56.438 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~205s 2024-03-04T19:10:56.438 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~205s 2024-03-04T19:10:56.438 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~205s 2024-03-04T19:10:56.438 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~205s 2024-03-04T19:10:56.438 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~205s 2024-03-04T19:10:56.438 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~205s 2024-03-04T19:10:56.438 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~205s 2024-03-04T19:10:56.439 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~205s 2024-03-04T19:11:01.439 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~210s 2024-03-04T19:11:01.440 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~210s 2024-03-04T19:11:01.440 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~210s 2024-03-04T19:11:01.440 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~210s 2024-03-04T19:11:01.440 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~210s 2024-03-04T19:11:01.440 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~210s 2024-03-04T19:11:01.440 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~210s 2024-03-04T19:11:01.440 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~210s 2024-03-04T19:11:01.440 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~210s 2024-03-04T19:11:01.441 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~210s 2024-03-04T19:11:01.441 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~210s 2024-03-04T19:11:01.441 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~210s 2024-03-04T19:11:01.441 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~210s 2024-03-04T19:11:01.441 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~210s 2024-03-04T19:11:06.443 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~215s 2024-03-04T19:11:06.443 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~215s 2024-03-04T19:11:06.443 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~215s 2024-03-04T19:11:06.443 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~215s 2024-03-04T19:11:06.443 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~215s 2024-03-04T19:11:06.443 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~215s 2024-03-04T19:11:06.443 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~215s 2024-03-04T19:11:06.444 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~215s 2024-03-04T19:11:06.444 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~215s 2024-03-04T19:11:06.444 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~215s 2024-03-04T19:11:06.444 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~215s 2024-03-04T19:11:06.444 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~215s 2024-03-04T19:11:06.444 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~215s 2024-03-04T19:11:06.444 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~215s 2024-03-04T19:11:11.445 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~220s 2024-03-04T19:11:11.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~220s 2024-03-04T19:11:11.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~220s 2024-03-04T19:11:11.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~220s 2024-03-04T19:11:11.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~220s 2024-03-04T19:11:11.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~220s 2024-03-04T19:11:11.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~220s 2024-03-04T19:11:11.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~220s 2024-03-04T19:11:11.447 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~220s 2024-03-04T19:11:11.447 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~220s 2024-03-04T19:11:11.447 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~220s 2024-03-04T19:11:11.447 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~220s 2024-03-04T19:11:11.447 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~220s 2024-03-04T19:11:11.447 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~220s 2024-03-04T19:11:16.448 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~225s 2024-03-04T19:11:16.449 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~225s 2024-03-04T19:11:16.449 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~225s 2024-03-04T19:11:16.449 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~225s 2024-03-04T19:11:16.449 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~225s 2024-03-04T19:11:16.449 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~225s 2024-03-04T19:11:16.449 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~225s 2024-03-04T19:11:16.449 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~225s 2024-03-04T19:11:16.449 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~225s 2024-03-04T19:11:16.449 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~225s 2024-03-04T19:11:16.450 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~225s 2024-03-04T19:11:16.450 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~225s 2024-03-04T19:11:16.450 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~225s 2024-03-04T19:11:16.450 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~225s 2024-03-04T19:11:21.451 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~230s 2024-03-04T19:11:21.451 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~230s 2024-03-04T19:11:21.451 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~230s 2024-03-04T19:11:21.451 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~230s 2024-03-04T19:11:21.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~230s 2024-03-04T19:11:21.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~230s 2024-03-04T19:11:21.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~230s 2024-03-04T19:11:21.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~230s 2024-03-04T19:11:21.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~230s 2024-03-04T19:11:21.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~230s 2024-03-04T19:11:21.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~230s 2024-03-04T19:11:21.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~230s 2024-03-04T19:11:21.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~230s 2024-03-04T19:11:21.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~230s 2024-03-04T19:11:26.454 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~235s 2024-03-04T19:11:26.454 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~235s 2024-03-04T19:11:26.454 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~235s 2024-03-04T19:11:26.454 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~235s 2024-03-04T19:11:26.454 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~235s 2024-03-04T19:11:26.454 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~235s 2024-03-04T19:11:26.455 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~235s 2024-03-04T19:11:26.455 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~235s 2024-03-04T19:11:26.455 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~235s 2024-03-04T19:11:26.455 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~235s 2024-03-04T19:11:26.455 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~235s 2024-03-04T19:11:26.455 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~235s 2024-03-04T19:11:26.455 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~235s 2024-03-04T19:11:26.455 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~235s 2024-03-04T19:11:31.456 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~240s 2024-03-04T19:11:31.456 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~240s 2024-03-04T19:11:31.456 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~240s 2024-03-04T19:11:31.457 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~240s 2024-03-04T19:11:31.457 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~240s 2024-03-04T19:11:31.457 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~240s 2024-03-04T19:11:31.457 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~240s 2024-03-04T19:11:31.457 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~240s 2024-03-04T19:11:31.457 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~240s 2024-03-04T19:11:31.457 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~240s 2024-03-04T19:11:31.457 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~240s 2024-03-04T19:11:31.457 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~240s 2024-03-04T19:11:31.457 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~240s 2024-03-04T19:11:31.458 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~240s 2024-03-04T19:11:36.459 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~245s 2024-03-04T19:11:36.459 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~245s 2024-03-04T19:11:36.459 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~245s 2024-03-04T19:11:36.459 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~245s 2024-03-04T19:11:36.460 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~245s 2024-03-04T19:11:36.460 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~245s 2024-03-04T19:11:36.460 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~245s 2024-03-04T19:11:36.460 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~245s 2024-03-04T19:11:36.460 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~245s 2024-03-04T19:11:36.460 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~245s 2024-03-04T19:11:36.460 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~245s 2024-03-04T19:11:36.460 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~245s 2024-03-04T19:11:36.460 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~245s 2024-03-04T19:11:36.461 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~245s 2024-03-04T19:11:41.461 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~250s 2024-03-04T19:11:41.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~250s 2024-03-04T19:11:41.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~250s 2024-03-04T19:11:41.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~250s 2024-03-04T19:11:41.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~250s 2024-03-04T19:11:41.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~250s 2024-03-04T19:11:41.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~250s 2024-03-04T19:11:41.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~250s 2024-03-04T19:11:41.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~250s 2024-03-04T19:11:41.463 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~250s 2024-03-04T19:11:41.463 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~250s 2024-03-04T19:11:41.463 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~250s 2024-03-04T19:11:41.463 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~250s 2024-03-04T19:11:41.463 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~250s 2024-03-04T19:11:46.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~255s 2024-03-04T19:11:46.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~255s 2024-03-04T19:11:46.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~255s 2024-03-04T19:11:46.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~255s 2024-03-04T19:11:46.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~255s 2024-03-04T19:11:46.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~255s 2024-03-04T19:11:46.466 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~255s 2024-03-04T19:11:46.466 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~255s 2024-03-04T19:11:46.466 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~255s 2024-03-04T19:11:46.466 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~255s 2024-03-04T19:11:46.466 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~255s 2024-03-04T19:11:46.466 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~255s 2024-03-04T19:11:46.466 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~255s 2024-03-04T19:11:46.466 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~255s 2024-03-04T19:11:51.468 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~260s 2024-03-04T19:11:51.468 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~260s 2024-03-04T19:11:51.469 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~260s 2024-03-04T19:11:51.469 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~260s 2024-03-04T19:11:51.469 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~260s 2024-03-04T19:11:51.469 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~260s 2024-03-04T19:11:51.469 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~260s 2024-03-04T19:11:51.469 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~260s 2024-03-04T19:11:51.469 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~260s 2024-03-04T19:11:51.469 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~260s 2024-03-04T19:11:51.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~260s 2024-03-04T19:11:51.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~260s 2024-03-04T19:11:51.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~260s 2024-03-04T19:11:51.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~260s 2024-03-04T19:11:56.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~265s 2024-03-04T19:11:56.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~265s 2024-03-04T19:11:56.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~265s 2024-03-04T19:11:56.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~265s 2024-03-04T19:11:56.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~265s 2024-03-04T19:11:56.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~265s 2024-03-04T19:11:56.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~265s 2024-03-04T19:11:56.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~265s 2024-03-04T19:11:56.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~265s 2024-03-04T19:11:56.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~265s 2024-03-04T19:11:56.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~265s 2024-03-04T19:11:56.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~265s 2024-03-04T19:11:56.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~265s 2024-03-04T19:11:56.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~265s 2024-03-04T19:12:01.475 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~270s 2024-03-04T19:12:01.475 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~270s 2024-03-04T19:12:01.475 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~270s 2024-03-04T19:12:01.475 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~270s 2024-03-04T19:12:01.475 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~270s 2024-03-04T19:12:01.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~270s 2024-03-04T19:12:01.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~270s 2024-03-04T19:12:01.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~270s 2024-03-04T19:12:01.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~270s 2024-03-04T19:12:01.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~270s 2024-03-04T19:12:01.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~270s 2024-03-04T19:12:01.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~270s 2024-03-04T19:12:01.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~270s 2024-03-04T19:12:01.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~270s 2024-03-04T19:12:06.478 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~275s 2024-03-04T19:12:06.478 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~275s 2024-03-04T19:12:06.478 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~275s 2024-03-04T19:12:06.478 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~275s 2024-03-04T19:12:06.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~275s 2024-03-04T19:12:06.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~275s 2024-03-04T19:12:06.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~275s 2024-03-04T19:12:06.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~275s 2024-03-04T19:12:06.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~275s 2024-03-04T19:12:06.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~275s 2024-03-04T19:12:06.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~275s 2024-03-04T19:12:06.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~275s 2024-03-04T19:12:06.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~275s 2024-03-04T19:12:06.480 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~275s 2024-03-04T19:12:11.480 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~280s 2024-03-04T19:12:11.481 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~280s 2024-03-04T19:12:11.481 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~280s 2024-03-04T19:12:11.481 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~280s 2024-03-04T19:12:11.481 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~280s 2024-03-04T19:12:11.481 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~280s 2024-03-04T19:12:11.481 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~280s 2024-03-04T19:12:11.481 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~280s 2024-03-04T19:12:11.481 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~280s 2024-03-04T19:12:11.481 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~280s 2024-03-04T19:12:11.482 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~280s 2024-03-04T19:12:11.482 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~280s 2024-03-04T19:12:11.482 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~280s 2024-03-04T19:12:11.482 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~280s 2024-03-04T19:12:16.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~285s 2024-03-04T19:12:16.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~285s 2024-03-04T19:12:16.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~285s 2024-03-04T19:12:16.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~285s 2024-03-04T19:12:16.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~285s 2024-03-04T19:12:16.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~285s 2024-03-04T19:12:16.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~285s 2024-03-04T19:12:16.484 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~285s 2024-03-04T19:12:16.484 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~285s 2024-03-04T19:12:16.484 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~285s 2024-03-04T19:12:16.484 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~285s 2024-03-04T19:12:16.484 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~285s 2024-03-04T19:12:16.484 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~285s 2024-03-04T19:12:16.484 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~285s 2024-03-04T19:12:21.485 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~290s 2024-03-04T19:12:21.485 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~290s 2024-03-04T19:12:21.485 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~290s 2024-03-04T19:12:21.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~290s 2024-03-04T19:12:21.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~290s 2024-03-04T19:12:21.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~290s 2024-03-04T19:12:21.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~290s 2024-03-04T19:12:21.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~290s 2024-03-04T19:12:21.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~290s 2024-03-04T19:12:21.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~290s 2024-03-04T19:12:21.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~290s 2024-03-04T19:12:21.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~290s 2024-03-04T19:12:21.486 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~290s 2024-03-04T19:12:21.487 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~290s 2024-03-04T19:12:26.487 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~295s 2024-03-04T19:12:26.488 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~295s 2024-03-04T19:12:26.488 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~295s 2024-03-04T19:12:26.488 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~295s 2024-03-04T19:12:26.488 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~295s 2024-03-04T19:12:26.488 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~295s 2024-03-04T19:12:26.488 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~295s 2024-03-04T19:12:26.488 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~295s 2024-03-04T19:12:26.488 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~295s 2024-03-04T19:12:26.488 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~295s 2024-03-04T19:12:26.489 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~295s 2024-03-04T19:12:26.489 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~295s 2024-03-04T19:12:26.489 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~295s 2024-03-04T19:12:26.489 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~295s 2024-03-04T19:12:31.490 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~300s 2024-03-04T19:12:31.490 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~300s 2024-03-04T19:12:31.490 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~300s 2024-03-04T19:12:31.490 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~300s 2024-03-04T19:12:31.490 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~300s 2024-03-04T19:12:31.490 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~300s 2024-03-04T19:12:31.490 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~300s 2024-03-04T19:12:31.491 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~300s 2024-03-04T19:12:31.491 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~300s 2024-03-04T19:12:31.491 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~300s 2024-03-04T19:12:31.491 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~300s 2024-03-04T19:12:31.491 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~300s 2024-03-04T19:12:31.491 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~300s 2024-03-04T19:12:31.491 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~300s 2024-03-04T19:12:31.491 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-03-05T05:25:30.105 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-03-05T05:25:30.140 DEBUG:teuthology.task.console_log:Killing console logger for smithi137 2024-03-05T05:25:30.141 DEBUG:teuthology.task.console_log:Killing console logger for smithi173 2024-03-05T05:25:30.141 DEBUG:teuthology.exit:Finished running handlers